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SOF  Language  and  Culture  Needs  Assessment  Project 


Defense  Language  Proficiency  Test  (DLPT) 


EXECUTIVE  SUMMARY 


Moderator: 

Participant  1: 
Participant  2: 
Participant  3: 


“ ...so  the  DLPT  tests  listening  and  reading.  Is  that  the  appropriate  test 
for  what  you  do  on  the  job?” 

“No.  ” 

“No.  ” 

“No,  not  at  all.  ” 

19th  Special  Forces  Group  (SFG)  Focus  Group 


This  report  examines  perceptions  of  the  Defense  Language  Proficiency  Test  (DLPT)  in  the  Special 
Operations  Forces  (SOF)  community.  Specifically,  this  report  documents  SOF  leader  support  of  DLPT 
language  testing,  SOF  operator  DLPT  testing  experiences,  and  SOF  operator  and  leader  views  of  the 
DLPT.  Overall,  this  report  highlights  three  main  concerns  that  the  United  States  Special  Operations 
Command  (USSOCOM)  and  SOF  component  leadership  should  address  as  appropriate: 


•  Responses  indicate  that  SOF  leaders  do  not  fully  support  the  use  of  the  DLPT  for  language 
proficiency  testing  in  the  SOF  community.  It  is  important  to  determine  whether  this  lack  of 
support  is  specific  to  the  DLPT  because  it  lacks  mission  relevance,  or  whether  these  findings 
reflect  a  lack  of  support  for  language  testing  in  general.  A  general  lack  of  support  (i.e.,  not  DLPT 
specific)  could  have  implications  for  language-specific  mission  readiness.  USSOCOM  should 
assess  support  for  language  testing  in  the  future  once  the  Oral  Proficiency  Interview  (OPI)  has 
been  the  test  of  record  for  language  proficiency  for  a  few  years.  Since  SOF  personnel  perceive 
the  OPI  as  more  job  relevant1,  SOF  community  support  for  the  language  testing  process  should 
improve. 


•  Almost  half  of  SOF  operators  reported  experiencing  scheduling  and  technical  problems  during 
DLPT  testing.  Issues  with  test  availability/scheduling,  test  delivery  and  the  testing  environment 
are  serious  and  may  affect  the  reliability  and  validity  of  DLPT  scores  and,  therefore,  the  number 
of  SOF  personnel  meeting  language  testing  requirements. 

•  SOF  operators  and  leaders  indicated  that  the  DLPT  does  not  assess  the  language  proficiency  skills 
needed  for  missions.  SOF  leaders  and  operators  indicate  speaking  is  the  most  relevant  language 
skill  for  their  missions.2  This  is  an  issue  of  alignment  of  testing  with  needed  mission  capability  - 
the  DLPT  lacks  alignment  with  the  skills  required  on  SOF  missions  and,  therefore,  may  interfere 
with  development  of  needed  skills. 


1  See  Oral  Proficiency  Interview  (Technical  Report  #201001 1006)  for  additional  findings  related  to  SOF  operator 
and  leader  reactions  to  USSOCOM’s  recent  standard  change. 

2  Inside  AOR  Use  of  Language  (Technical  Report  #201001 1010)  and  Outside  AOR  Use  of  Language  (Technical 
Report  #2010011011). 
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These  criticisms  have  not  been  resolved  with  newer  versions  of  the  DLPT  as  SOF  operators  and  leaders 
still  react  negatively  to  the  DLPT5  (e.g.,  indicated  that  the  test  is  too  difficult  and  is  not  a  valid 
assessment).  Therefore,  the  findings  presented  in  this  report  support  the  move  away  from  using  the  DLPT 
as  the  test  of  record  for  language  proficiency  in  the  SOF  community. 

SOF  Leader  Support  for  DLPT  Testing 

As  part  of  their  leadership  role,  SOF  leaders  should  monitor  whether  SOF  operators  fulfill  USSOCOM’s 
language  testing  requirement  and  encourage  them  to  do  so.3  At  the  time  of  the  survey,  the  test  of  record 
was  the  DLPT.  Of  SOF  leaders  in  units  that  require  personnel  to  take  the  DLPT  and  who  could  comment 
on  DLPT  testing  (n  =  371),  51%  (n  =  183)  indicated  their  units  fell  below  USSOCOM’s  requirement  of 
more  than  80%  of  unit  personnel  meeting  language  testing  requirements.  Therefore,  approximately  half 
of  SOF  units  did  not  meet  USSOCOM’s  standard  and  should  be  following  mandated  language  training 
programs.4 

Low  percentages  of  SOF  units  meeting  testing  requirements  may  be  due  to  the  large  portions  of  SOF 
leaders  who  reported  not  paying  attention  to  DLPT  scores,  not  believing  DLPT  ratings  are  important,  and 
not  encouraging  SOF  operators  to  stay  up-to-date  on  language  testing  or  do  well  on  the  DLPT.  For 
example,  most  SOF  leaders  reported  that  the  DLPT  was  not  related  or  only  slightly  related  to  what  SOF 
operators  do  on  the  job  and  many  SOF  leaders  commented  that  DLPT  content  is  unrelated  to  the  mission, 
job,  or  military.  These  results  show  a  lack  of  command  support  for  DLPT  language  testing,  which  was 
also  found  in  the  2004  SOF  Language  Transformation  Needs  Assessment  (Surface,  Poncheri,  Lemmond 
&  Shetye,  2005). 5  This  lack  of  support  is  likely  related  to  the  DLPT’s  lack  of  relevance  for  the  SOF 
operator  (Surface,  Poncheri,  Lemmond  &  Shetye,  2005). 

SOF  Operator  Experiences  with  the  DLPT 

Since  many  SOF  operators  reported  taking  the  DLPT  (93%,  n  =  1,086),  it  is  important  to  identify  whether 
they  encounter  problems  during  testing  and,  if  so,  the  types  of  problems  experienced.  During  their  most 
recent  DLPT,  44%  ( n  =  468)  of  SOF  operators  reported  experiencing  problems.  The  most  commonly 
reported  problems  included  computer/technical  issues  (24%,  n  =  257)  and  test  scheduling  delays  (22%,  n 
=  235). 


(( Computer  would  freeze  up  during  test.  ” 


SOF  Operator,  95th  CA  Bde 


3  USSOCOM  recently  changed  their  foreign  language  testing  requirement  to  reflect  the  importance  of  speaking  and 
listening  skills  (as  measured  by  a  two-skill  OPI).  However,  at  the  time  of  this  survey,  many  respondents  reported 
being  from  units  that  required  the  DLPT  for  language  testing. 

4  However,  these  are  only  SOF  leader  estimates  rather  than  actual  counts.  If  there  is  a  discrepancy  between  these 
estimates  and  actual  counts,  the  discrepancy  suggests  that  SOF  leaders  are  not  paying  close  attention  to  SOF 
operator  language  test  scores. 

5  See  Command  Support  for  Language:  Grading  the  Chain  of  Command  Technical  Report  #201001 1006)  for 
additional  findings  related  to  command  support  for  language  testing. 
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“LAST  4  SCHEDULED  APPOINTMENTS  WERE  CANCELED  AND  CURRENT  "RE¬ 
SCHEDULED"  APPOINTMENT  WAS  FOR  A  DATE  8  WEEKS  OUT” 

SOF  Operator,  5th  SFG 

Computer/technical  issues  must  be  addressed  since  they  may  have  a  negative  impact  on  performance, 
which  could  misrepresent  SOF  proficiency  and  force  capability  estimates.  Specifically,  technical  issues 
can  impact  the  reliability  and  validity  of  test  scores  and,  in  extreme  cases,  the  validity  as  well.  Test 
scheduling  delays  should  also  be  addressed  as  they  may  be  affect  SOF  operators’  ability  to  stay  up-to-date 
on  their  language  testing  requirement.  Additional  issues  that  were  less  frequently  mentioned  included 
problems  accessing  testing  centers,  delays  or  problems  receiving  feedback,  and  disruptions  while  testing. 

“too  many  people  in  test  area.. ..most  of  them  are  the  test  takers...  should  limit  the  amount 
of  people.  ” 

SOF  Operator,  95th  CA  Bde 

Although  outside  the  direct  control  of  SOF,  SOF  leaders  can  raise  these  issues  with  those  who  have 
propensity  and  oversight  for  testing  and  test  delivery. 

Perceptions  of  DLPT  Accuracy  and  Job-relatedness 


“I  think  the  first  thing  needs  to  be  addressed  is  the  DLPT \  [...]  It  does  not  meet  our  needs 
whatsoever. 

Air  Force  Special  Operations  Command  (AFSOC)  Focus  Group 

Recently,  USSOCOM  changed  its  test  of  record  for  language  proficiency  from  the  DLPT  (reading  and 
listening)  to  the  two-skill  OPI.  Findings  support  this  change  as  many  SOF  operators  and  leaders  indicated 
that  the  DLPT  neither  relates  to  their  jobs  nor  accurately  assesses  language  proficiency. 

“ ...even  the  best  students,  who  were  able  to  communicate  clearly  in  Russian,  did  not 
achieve  a  high  score  on  the  DLPT” 

SOF  Operator,  4th  MISG 

Therefore,  the  move  away  from  testing  on  the  DLPT  to  OPI  testing  supports  USSOCOM’s  goals 
of  emphasizing  mission-related  language  skills.6 

Perceptions  of  DLPT  accuracy  have  not  improved  with  the  DLPT5.  In  fact,  perceptions  have  gotten  more 
negative  as  SOF  operators  who  last  tested  on  the  DLPT  IV  viewed  the  DLPT  as  slightly  more  accurate 
than  those  who  last  took  the  DLPT5. 


6  See  Oral  Proficiency  Interview  (OPI;  Technical  Report  #201001 1016)  for  more  information  on  SOF  operator  and 
leader  perspectives  on  the  standard  change. 
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"...  the  DLPT  V  is  a  test  designed  for  DLI-trained  linguists  and  interpreters... not  Special 
Forces  operators.  It  is  entirely  too  unrelated  to  the  conversations/situations  that  an 
operator  will  typically  encounter.  ” 

SOF  Operator,  7th  SFG 

Additionally,  findings  mirror  those  found  during  the  2004  SOF  Language  Transformation  Needs 
Assessment  (Surface,  Poncheri,  Lemmond  &  Shetye,  2005).  When  responding  to  that  survey,  SOF 
operators  and  leaders  indicated  that  the  DLPT  did  not  accurately  measure  language  proficiency  or  how 
operators  use  language  on  missions.  Moreover,  recent  research  within  the  SOF  community7  has  indicated 
that  DLPT  listening  is  not  an  effective  proxy  for  speaking  proficiency,  which  is  the  desired  skill  for  most 
SOF.  Combining  past  research  with  the  current  study,  it  is  evident  that  the  DLPT  does  not  meet  the 
testing  needs  of  the  SOF  community  and  does  not  facilitate  the  alignment  of  capability  with  mission 
requirements.  However,  many  in  SOF  will  still  take  the  DLPT  to  qualify  for  Foreign  Language 
Proficiency  Bonus  (FLPB),  make  this  problematic  for  the  SOF  community  and  the  goal  of  developing 
mission-relevant  foreign  language  capability.  USSOCOM  and  SOF  component  leadership  should  take 
steps  to  address  these  concerns  to  better  align  foreign  language  proficiency  testing  with  the  needs  of  the 
SOF  community. 

See  Appendix  A  for  details  on  the  2009  SOF LCNA  Project.  For  questions  or  more  information  about  the 
SOFLO  and  this  project,  please  contact  Mr.  Jack  Donnelly  (i ohn. donnellv@socom. mil).  For  specific 
questions  related  to  data  collection  or  reports  associated  with  this  project,  please  contact  Dr.  Eric  A. 
Surface  (esur  face@s  wa-consulting.  com)  or  Dr.  Reanna  Poncheri  Harman  (rpharman@swa- 
consulting.com)  with  SWA  Consulting  Inc. 


7  Using  the  DLPT  as  a  Proxy  for  the  OPI:  Are  Reading  and  N on-Participatory  Listening  Scores  a  Substitute  for 
Direct  Assessment  of  Speaking  Proficiency?  (Technical  Report  #2010010624) 
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SECTION  Is  REPORT  AND  PROJECT  OVERVIEW 
Defense  Language  Proficiency  Test  (DLPT)  Report  Purpose 

Annual  language  proficiency  testing  has  a  central  role  in  monitoring  and  maintain  language  capability  in 
the  military  in  general  and,  specifically,  in  the  Special  Operations  Forces  (SOF)  community.  The  United 
States  Special  Operations  Command  (USSOCOM)  requires  SOF  personnel  to  demonstrate  they  have 
maintained  their  language  skills  through  annual  proficiency  testing,  which  provides  USSOCOM  and 
Special  Operations  Forces  Language  Office  (SOFLO)  information  on  current  SOF  language  capabilities. 
Unit  recently,  the  test  of  record  has  been  the  Defense  Language  Proficiency  Test  (DLPT)  for  SOF.  Since 
most  SOF  personnel  have  historically  taken  the  DLPT  and  may  still  take  the  DLPT  to  fulfill  their  annual 
language  testing  requirement  and  qualify  for  proficiency  pay,  their  experiences  with  and  perceptions  of 
the  test  need  to  be  examined.8  This  report  examines  SOF  leader  support  of  language  testing  (Section  II), 
SOF  operator  DLPT  testing  experiences  (Section  III),  and  SOF  operator  and  leader  perceptions  of  the 
DLPT’s  job-relatedness  and  accuracy  (Section  IV).  Appendix  A  details  the  2009  SOF LCN A  Project ,  and 
Appendix  B  discusses  the  report  methodology.  Appendices  C  through  H  provides  findings  by  component 
and  job  (e.g.,  SOF  operator,  SOF  leader,  MI  Linguists  attached  to  a  SOF  unit)  while  Appendix  I  provides 
detailed  comment  code  results.  Appendix  J  provides  all  comment  codes  used  in  the  report. 

LCNA  Project  Purpose 

The  Special  Operations  Forces  Language  Office  (SOFLO)  commissioned  the  2009  SOF  LCNA  Project  to 
gain  insights  on  language  and  culture  capability  and  issues  across  the  United  States  Special  Operations 
Command  (USSOCOM).  The  goal  of  this  organizational-level  needs  assessment  is  to  inform  strategy  and 
policy  to  ensure  SOF  personnel  have  the  language  and  culture  skills  needed  to  conduct  their  missions 
effectively.  Data  were  collected  between  March  and  November  2009  from  personnel  in  the  SOF 
community,  including  operators  and  leaders.  Findings,  gathered  via  focus  groups  and  a  web-based 
survey,  will  be  presented  in  a  series  of  reports  divided  into  three  tiers.  The  specific  reports  in  each  of 
these  tiers  will  be  determined  and  contracted  by  the  SOFLO.  Tier  I  reports  focus  on  specific,  limited 
issues  (e.g.,  Inside  AOR  Use  of  Language).  Tier  //reports  integrate  and  present  the  most  important 
findings  across  related  Tier  /reports  (e.g.,  Use  of  Language  and  Culture  on  Deployment)  while  including 
additional  data  and  analysis  on  the  topic.  One  Tier  III  report  presents  the  most  important  findings, 
implications,  and  recommendations  across  all  topics  explored  in  this  project.  The  remaining  Tier  III 
reports  present  findings  for  specific  SOF  organizations  [e.g.,  Air  Force  Special  Operations  Command 
(AFSOC),  Special  Forces  (SF)  Command].  Two  foundational  reports  document  the  methodology  and 
participants  associated  with  this  project.  Original  report  topics  were  determined  by  the  SOFLO. 

Relationship  of  Defense  Language  Proficiency  Test  (DLPT)  to  the  LCNA  Project 

Defense  Language  Proficiency  Test  (DLPT)  is  a  Tier  /  report  that  will  be  integrated  with  two  other  Tier  I 
reports,  Oral  Proficiency  Interview  ( OPI)  and  Defense  Language  Aptitude  Battery  (DLAB):  Perspectives 
from  the  Field,  into  a  Tier  II  report,  Testing/Metrics  (see  Appendix  A  for  the  planned  report  structure). 


8  The  other  test  sometimes  taken  to  fulfill  the  annual  requirement,  the  Oral  Proficiency  Interview  (OPI),  is  evaluated 
in  a  separate  report  [see  Oral  Proficiency  Interview  (OPI),  Technical  Report  #201001 1016]. 


11/12/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011015] 


Page  7 


SOF  Language  and  Culture  Needs  Assessment  Project 


Defense  Language  Proficiency  Test  (DLPT) 


SECTION  II:  SOF  LEADER  SUPPORT  FOR  DLPT  TESTING 

This  section  investigates  SOF  leaders’  support  for  DLPT  language  testing  through  their  reported  attitudes 
and  behaviors.  While  USSOCOM  recently  changed  their  test  of  record  for  language  proficiency  to  the 
OPI,  some  SOF  operators  still  tested  on  the  DLPT  at  the  time  of  this  survey  and  likely  still  continue  to 
test  on  the  DLPT  to  qualify  for  the  proficiency  bonus.  This  section  documents  the  extent  to  which  SOF 
leaders  in  units  that  required  SOF  operators  to  test  on  the  DLPT  support  language  testing  and,  by 
extension,  development  of  language  proficiency  in  their  units. 

Research  Questions 

•  How  many  SOF  leaders  report  that  personnel  in  their  unit  are  required  to  take  the  DLPT? 

•  According  to  SOF  leaders  in  units  that  require  SOF  operators  to  test  on  the  DLPT: 

o  What  percentage  of  SOF  operators  in  their  units  that  are  up-to-date  on  language  testing 
requirements? 

o  How  often  do  SOF  leaders  in  units  that  require  DLPT  testing  encourage  SOF  operators  in 
their  units  to  stay  up-to-date  on  their  language  testing  requirement? 

•  How  often  are  SOF  leaders  encouraging  SOF  operators  to  study  and  do  well  on  the  DLPT? 

•  How  many  SOF  leaders  pay  attention  to  SOF  operators’  DLPT  ratings? 

•  How  important  are  SOF  operators’  DLPT  ratings  to  SOF  leaders? 

Main  Findings 

SOF  leader  reports  of  their  attitudes  and  behaviors  suggest  they  do  not  fully  support  DLPT  language 
testing.  Eighty-nine  percent  ( n  =  679)  of  SOF  leaders  indicated  that  personnel  in  their  unit  were  required 
to  take  the  DLPT  (Figure  1,  p.  9).  However,  only  49%  (/t  =  371)  were  in  a  position  to  comment  on  their 
SOF  operators’  DLPT  testing  (Figure  1,  p.  9).  Moreover,  although  74%  {n  =  268)  reported  they  often  or 
very  often  encourage  their  unit’s  SOF  operators  to  stay  up-to-date  on  the  language  testing  requirement 
(Figure  2,  p.  10),  51%  ( n  =  183)  of  SOF  leaders  reported  that  less  than  80%  of  SOF  operators  in  their 
units  were  up-to-date  on  their  language  testing  requirement  (Figure  3,  p.  10).  USSOCOM  policy 
stipulates  that  components  with  less  than  80%  of  their  personnel  meeting  the  language  testing  requirement 
are  required  to  implement  mandated  training  hours  and  retest  upon  completion  of  training  (USSOCOM  M 
350-8,  2009).  This  suggests  that  a  large  number  of  units  may  need  mandated  training.  9 

Additionally,  SOF  leaders  in  a  position  to  comment  on  their  unit’s  DLPT  ratings,  61%  ( n  =  228)  reported 
that  DLPT  ratings  are  not  important  or  moderately  important  to  them  (Figure  4,  p.  11).  Furthermore,  74% 
( n  =  273)  of  SOF  leaders  reported  they  pay  attention  to  SOF  operator  DLPT  ratings  (Figure  5,  p.  1 1)  and 
53%  (n  =  195)  reported  they  often  or  very  often  encourage  their  unit’s  SOF  operators  to  study  and  do  well 
on  the  DLPT  (Figure  6,  p.  12).  While  USSOCOM  has  recently  changed  its  test  of  record  from  the  DLPT 
to  the  OPI,  findings  could  indicate  that  SOF  leaders  may  not  fully  support  language  testing  in  general. 

9  These  are  SOF  leader  estimates  rather  than  actual  counts.  If  there  is  a  discrepancy  between  these  estimates  and 
actual  counts,  the  discrepancy  suggests  that  SOF  leaders  are  not  paying  close  attention  to  SOF  operator  language  test 
scores. 
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Detailed  Findings 

SOF  Leader  Familiarity  with  DLPT  Testing 

While  most  SOF  leaders  (89%,  n  =  679)  reported  that  personnel  in  their  unit  were  required  to  take  the 
DLPT,  only  49%  ( n  =  371)  were  in  a  position  to  comment  on  DLPT  testing  (Figure  1,  p.  9).  This  report’s 
remaining  sections  only  include  these  SOF  leaders. 

SOF  Leader  Perspectives  on  Unit  Language  Testing  Status 

Eighty-nine  percent  ( n  =  679)  of  SOF  leaders  were  in  units  that  required  personnel  to  take  the  DLPT  for 
language  proficiency  testing  (Figure  1,  p.  9.  However,  only  49%  (n  =  371)  were  in  a  position  to  comment 
on  DLPT  testing  (Figure  1,  p.  9).  Even  though  75%  ( n  =  268)  of  those  SOF  leaders  reported  they  often  or 
very  often  encourage  their  unit’s  SOF  operators  to  stay  up-to-date  on  the  language  testing  requirement 
(Figure  2,  p.  10),  51%  ( n  =  183)  reported  that  less  than  80%  of  their  unit’s  SOF  operators  were  up-to-date 
on  their  language  testing  requirement  (Figure  3,  p.  10). 10  USSOCOM  policy  stipulates  that  components 
with  less  than  80%  of  their  personnel  meeting  the  language  testing  requirement  are  required  to  implement 
mandated  training  hours  and  retest  upon  completion  of  training  (USSOCOM  M  350-8,  2009).  This 
suggests  that  a  large  number  of  units  may  need  mandated  training. *  11 

Figure  1.  SOF  leader  perspective  on  unit  DLPT  testing  requirements 


■  My  unit  requires  personnel  to 
take  the  DLPT  and  I  am  in  a 
position  to  comment 

■  My  unit  requires  personnel  to 
take  the  DLPT  and  I  am  not  in 
a  position  to  comment 

■  My  unit  does  not  require 
personnel  to  take  the  DLPT 


Note.  SOF  Leaders:  n  =  760. 


10  See  Appendix  C  for  results  by  component  and  Army  SOF  type. 

11  These  are  SOF  leader  estimates  rather  than  actual  counts.  If  there  is  a  discrepancy  between  these  estimates  and 
actual  counts,  the  discrepancy  suggests  that  SOF  leaders  are  not  paying  close  attention  to  SOF  operator  language  test 
scores. 


11/12/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011015] 


Page  9 


SOF  Language  and  Culture  Needs  Assessment  Project 


Defense  Language  Proficiency  Test  (DLPT) 


Figure  2.  Encouragement  of  SOF  operators  to  stay  up-to-date  on  the  language  testing  requirement 


■  Never 

■  Seldom 
Sometimes 

■  Often 

■  Very  Often 

0%  20%  40%  60%  80%  100% 

Note.  SOF  Leaders:  n  =  361.  See  Appendix  C  for  additional  results. 


Figure  3.  Estimated  percentage  of  SOF  operators  up-to-date  on  language  testing  requirement 


■  SOF  Leaders 


Note.  SOF  Leaders:  n  =  365;  CLPMs:  n  =  10.  See  Appendix  C  for  additional  results. 

SOF  Leader  Support  of  DLPT  Language  Testing 

Seventy-five  percent  of  SOF  leaders  ( n  =  268)  reported  they  often  or  very  often  encourage  their  unit’s 
SOF  operators  to  stay  up-to-date  on  the  language  testing  requirement  (Figure  2,  p.  10).  While  SOF 
leaders  support  their  SOF  operators  in  meeting  USSOCOM’s  testing  requirement,  findings  suggest  they 
do  not  fully  support  DLPT  language  testing. 12  Only  39%  (n  =  143)  reported  that  DLPT  ratings  are 
important  or  very  important  to  them  (Figure  4,  p.  1 1).  Moreover,  74%  ( n  =  273)  of  SOF  leaders  reported 
they  pay  attention  to  SOF  operator  DLPT  ratings  (Figure  5,  p.  1 1)  and  53%  (n  =  195)  reported  they  often 
or  very  often  encourage  their  unit’s  SOF  operators  to  study  and  do  well  on  the  DLPT  (Figure  6,  p.  12). 
This  suggests  that  while  SOF  leaders  often  encourage  SOF  operators  to  stay  up-to-date  on  their  DLPT 
language  testing  requirements,  and  thus  meet  USSOCOM’s  requirements  for  mission  readiness,  they 
themselves  do  not  believe  language  is  important  for  SOF  mission  success. 

12  See  Command  Support  for  Language:  Grading  the  Chain  of  Command  [Technical  Report  #201001 1006]  for 
more  information  on  SOF  leader  support  for  meeting  mission-related  language  requirements. 
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SOF  leaders  who  pay  attention  indicated  that  SOF  operator  DLPT  ratings  are  more  important  (M=  3.4) 
than  SOF  leaders  who  reported  not  paying  attention  (M=  1.8;  Figure  4,  p.  1 1).  SOF  leaders  who  pay 
attention  also  indicated  they  more  often  encouraged  SOF  operators  to  study  and  do  well  on  the  DLPT  ( M 
=  3.7)  than  SOF  leaders  who  reported  not  paying  attention  (M=  2.4;  Figure  6,  p.  12).  In  addition,  SOF 
leaders  who  reported  that  DLPT  ratings  are  more  important  to  them  also  reported  they  more  often 
encouraged  their  SOF  operators  to  study  and  do  well  on  the  test  (r  =  0.55,  p  <  .01). 

Figure  4.  Importance  of  DLPT  ratings  to  SOF  leaders 

Overall 

Pays  Attention 

Does  Not  Pay  Attention 


Note.  Pays  Attention  =  SOF  leaders  who  indicated  they  pay  attention  to  SOF  operators’  DLPT  ratings;  Does  not  Pay  Attention  =  SOF  leaders 
who  indicated  they  do  not  pay  attention  to  SOF  operators’  DLPT  ratings.  Overall:  n  =  371 ;  Pays  Attention:  n  =  273;  Does  not  Pay  Attention:  n  = 
97. 

Figure  5.  SOF  leader  attention  to  SOF  operator  DLPT  ratings 


i  Not  Important 
Slightly  Important 
Moderately  Important 
Important 
Very  Important 


>% 


20%  40%  60%  80%  100% 


■  Pays  Attention 

■  Does  Not  Pay  Attention 


Note.  SOF  Leaders:  n  =  370. 
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Figure  6.  SOF  leader  encouragement  to  study  and  do  well  on  the  DLPT 


Overall 


Pays  Attention 


Does  Not  Pay  Attention 


0%  20%  40%  60%  80%  100% 


■  Never 

■  Seldom 
Sometimes 

■  Often 

■  Very  Often 


Note.  Pays  Attention  =  SOF  leaders  who  indicated  they  pay  attention  to  SOF  operators’  DLPT  ratings,  Does  not  Pay  Attention  =  SOF  leaders 
who  indicated  they  do  not  pay  attention  to  SOF  operators’  DLPT  ratings.  Overall:  n  =  370;  Pays  Attention:  n  =  272;  Does  not  Pay  Attention:  n  = 
97. 
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SECTION  III:  SOF  OPERATOR  EXPEREINCES  WITH  THE  DLPT 

This  section  reports  the  DLPT  testing  history  of  SOF  operators  who  completed  the  survey.  In  addition, 
this  section  identifies  problems  SOF  operators  experienced  during  DLPT  testing  so  they  can  be 
communicated  to  the  Defense  Language  Institute  (DLI),  the  testing  authorities  for  Department  of 
Defense,  to  be  addressed. 

Research  Questions 

•  How  many  SOF  operators  have  taken  a  DLPT? 

•  What  problems  did  SOF  operators  experience  during  DLPT  testing? 

Main  Findings 

Nearly  all  SOF  operators  have  taken  a  DLPT  (93%,  n  =  1,086)  with  44%  ( n  =  468)  reporting  problems. 
Forty-four  percent  of  SOF  operators  report  the  DLPT5  as  their  most  recent  DLPT  test  version.  The  most 
commonly  reported  problems  included  computer/technical  issues  (24%,  n  =  257)  and  test  scheduling 
delays  (22%,  n  =  235).  Few  SOF  operators  reported  problems  accessing  testing  centers  (10%,  n  =  107), 
receiving  feedback  on  their  score  (8%,  n  =  88),  disruptions  while  testing  (8%,  n  =  85),  or  “other” 
problems  (5%,  n  =  48). 

Detailed  Findings 

SOF  Operator  DLPT  Testing  Background 

Most  SOF  operators  have  DLPT  testing  experience  (93%,  n  =  1,086),  with  many  testing  on  the  most 
current  form  (DLPT5:  44%,  n  =  467;  Figure  7,  p.  13)  within  the  year  prior  to  the  survey  (2009:  53%,  n  = 
558;  Figure  8,  p.  14). 13 

Figure  7.  SOF  operators’  most  recent  DLPT  version 


(Web) 

Note.  SOF  Operators:  n  =  1,064.  Figure  does  not  include  SOF  operators  who  did  not  know  which  version  they  last  took  (22%). 


13  The  remaining  sections  of  this  report  only  include  responses  from  SOF  operators  who  have  taken  a  DLPT. 
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Figure  8.  Date  of  SOF  operators’  most  recent  DLPT 


Note.  SOF  Operators:  n  =  1,053.  Figure  does  not  include  SOF  operators  who  last  tested  prior  to  2006  (6%). 

Problems  Experienced  During  DLPT  Testing 

While  slightly  over  half  of  SOF  operators  have  not  encountered  problems  during  DLPT  testing  (56%,  n  = 
603),  a  large  enough  portion  have  (44%,  n  =  468)  to  justify  immediate  attention  to  address  the  most 
common  problems  reported  by  survey  respondents  (Table  1,  p.  15)  and  focus  group  participants  (Table  2, 
p.  15).  SOF  operators  who  experienced  problems  (44%,  n  =  468)  most  often  encountered 
computer/technical  issues  (24%,  n  =  257)  and  test  scheduling  issues  (22%,  n  =  235;  Table  1,  p.  15). 

Some  SOF  operators  reported  additional  problems  during  testing,  including  problems  accessing  testing 
centers  (10%,  n  =  107),  delays/problems  receiving  feedback  on  their  score  (8%,  n  =  88),  and  disruptions 
while  testing  (8%,  n  =  85;  Table  1,  p.  15).  A  few  SOF  operators  reported  “other”  problems  (5%,  n  =  48), 
such  as  problems  with  the  test  content,  format  or  protocol  (2%,  n  =  17);  poor  audio  quality  (1%,  n  =  8); 
and  testing  environment  problems  (1%,  n  =  8;  Table  1,  p.  15). 
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Table  1.  Problems  experienced  during  DLPT  testing 


Reported  Testing  Problems 

Percentage  of  Total  Responses 

None 

56% 

Computer/technical  issues 

24% 

Test  scheduling  delays 

22% 

Problems  accessing  testing  centers 

10% 

Delays/problems  receiving  feedback  on  your  score 

8% 

Disruptions  while  taking  test 

8% 

Other 

5% 

“Other”  Themes 

Percentage  of  Total  Responses 

Test  content/format/protocol  problems 

2% 

Poor  audio  quality 

1% 

Testing  environment  problems 

1% 

DLPT  is  not  job  relevant 

<  1% 

Not  able  to  prepare  for  the  test 

<  1% 

General  negative 

<  1% 

Scheduling  conflicts 

<  1% 

Scoring  system  is  not  fair/accurate 

<  1% 

Total  SOF  Operators  who  Responded  to  These  Items 

1,071 

Note.  Testing  problems  presented  in  italics  were  coded  from  responses  to  the  “other”  response  category.  Percentages  may  not  sum  to  100%  as 


respondents  could  select  or  comment  on  more  than  one  problem. 


Table  2.  Focus  group  themes 


Testing  Issues  (Administrative) 

Number  of  Segments 

Computer-technical  issues 

6 

Disruptions  while  taking  the  test 

2 

Issues  receiving  test  results 

1 

Difficulty  scheduling  test 

0 

SOF  operators’  comments  expand  upon  the  problems  they  experienced  during  DLPT  testing  (Table  3,  p. 
16).  The  most  frequently  elaborated  upon  comment  involved  technical  problems,  scheduling  problems, 
and  test  content/format/protocol. 
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Table  3.  Elaborations  of  problems  during  DLPT  testing 


Themes 

Percentage  of  Total  Comments 

Test  content/format/protocol  is  bad/not  appropriate 

17% 

Technical  problems 

16% 

Scheduling  conflicts 

14% 

DLPT  is  not  an  accurate  assessment 

10% 

Testing  environment  problems 

8% 

DLPT  is  not  job  relevant 

8% 

Leadership/policy  problems  related  to  DLPT  testing 

6% 

DLPT  is  too  difficult 

6% 

General  negative 

5% 

Training  does  not  match  what  is  tested  on  the  DLPT 

5% 

Delays  getting  scores 

3% 

Not  receiving  test  feedback  for  improvement 

1% 

Test  content/format/protocol  is  good/appropriate 

1% 

Total  Comments 

143 

Note.  Percentages  may  not  sum  to  100%  as  respondents  could  comment  on  more  than  one  topic.  See  Appendix  J  for  definitions  and  example 
comments  for  each  theme.  Table  does  not  include  comments  coded  as  “none/na/not  relevant  response”  (22%). 


Commonly  mentioned  problems  with  the  test  content,  format  and  protocol  included  perceptions  that  the 
test’s  questions  and/or  response  options  were  too  subjective,  unclear  and  confusing  questions,  test  length, 
and  problems  understanding  the  English  (rather  than  the  target  language)  used  on  the  test: 

‘‘The  problem  with  the  DLPT5  is  not  with  the  Arabic  passages,  it  is  with  the  English 
questions.  I  am  a  native  English  speaker  with  a  college  degree  and  I  did  not  understand 
what  the  questions  were  asking  me  in  English.  ” 

SOF  Operator,  5th  SFG 

The  technical  problems  SOF  operators  experienced  often  resulted  from  computers  shutting  down  or 
freezing  up,  equipment  malfunctions,  or  poor  audio  quality  for  the  listening  samples: 

“ Also  I  want  to  say  the  computers  itself,  [...]  probably  broke  down  six  times  [...].  I  don ’t 
know  what  happened,  if  it  was  just  an  issue  or  something  was  going  on  at  the  time.  ” 

Naval  Special  Warfare  Command  (WARCOM)  Focus  Group 

“THE  VOLUME  WAS  NOT  ADJUSTABLE  AND  MY  HEADPHONES  DID  NOT  WORK! 

I  COULD  NOT  HEAR  THE  SCENARIOS. 99 

SOF  Operator,  4th  Military  Information  Support  Group  (MISG)14 

The  scheduling  conflicts  SOF  operators  encountered  included  limited  availability  of  testing  locations, 
problems  with  the  testing  centers  and  staff,  and  difficulty  scheduling  testing  due  to  other  requirements: 


14  Formerly  4th  Psychological  Operations  Group  (POG). 
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“There  is  a  back  log  at  Ft.  Bragg  to  take  the  DLPT.  This  is  an  issue  because  you  are 
required  to  schedule  a  test  at  least  4  months  in  advance.  In  which  time  there  may  other 
requirements  and  the  soldier  might  require  to  fill  therefore  not  being  able  to  take  the 
test.  ” 

SOF  Operator,  7th  SFG 

“At  Fort  Lewis,  we  have  experienced  testing  center  contractor  issues,  where  soldiers 
have  shown  up  for  their  respective,  assigned  tests,  and  have  been  told  that  they  have  been 
dropped  because  the  contracted  staff  are  on  strike/unavailable/dearth/etc.  ” 

SOF  Operator,  1st  SFG 

Of  the  remaining  comment  themes,  some  were  related  to  the  accuracy  or  validity  of  the  DLPT.  For 
example  10%  of  the  comments  made  indicated  that  the  DLPT  is  not  an  accurate  assessment,  while  8%  of 
the  comments  indicated  that  the  DLPT  is  not  job-related. 

“I  always  say  you  can  probably  score  an  high  score  on  the  DLPT  but  would  that  make 
you  more  effected  on  the  mission  I  don  't  think  so,  the  DLPT  should  be  graded  and  also 
pay  by  the  skills  that  the  individuals  have  in  speaking  the  language  not  on  what  he  scores 
on  the  test,  because  lots  of  SM score  very  good  on  this  test  and  doesn't  even  know  how  to 
speak  the  language  at  all  so  with  this  being  say  how  that  person  can  help  on  the 
mission.  ” 

SOF  Operator,  4thMISG 

An  additional  8%  of  the  comments  were  related  to  problems  in  the  testing  environment,  while  6% 
identified  leadership/policy  problems  related  to  testing. 

“Poor  Testing  environment  with  no  AC  in  the  spring  or  summer” 

SOF  Operator,  95th  CA  Bde 

A  small  percentage  (4%)  were  related  to  getting  feedback  either  through  receipt  of  scores  in  a  timely 
manner  (3%)  or  related  to  improvements  in  future  testing  (1%).  Finally,  only  one  SOF  operator 
commented  positively  on  DLPT  testing,  mentioning  that  the  test  content,  format,  and  protocol  were  good. 
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SECTION  IV:  PERCEPTIONS  OF  DLPT  ACCURACY  AND  JOB-RELATEDNESS 

This  section  describes  SOF  operator  and  leader  perceptions  of  the  DLPT,  particularly  perceptions  of  its 
job-relatedness  and  accuracy. 15  Additionally,  this  section  presents  comments  from  survey  respondents 
and  focus  group  participants  on  general  feedback  related  to  the  DLPT.16  Since  DLPT  test  development 
and  test  administration  are  on-going  processes,  this  feedback  can  be  used  by  the  Defense  Language 
Institute  to  improve  future  test  administrations  and  test  versions. 

Research  Questions 

•  Do  SOF  operators  and  leaders  perceive  the  DLPT  as  a  job-related  and  accurate  assessment? 

•  What  feedback  do  survey  respondents  and  focus  group  participants  have  related  to  DLPT  testing? 

Main  Findings 

SOF  operators  and  leaders  do  not  view  the  DLPT  as  a  job-related  and  accurate  assessment  (Figures  9-10, 
pp.  19).  They  believe  that  the  DLPT  does  not  cover  all  the  language  tasks  SOF  operators  need  to  perform 
when  deployed  (Figure  1 1,  p.  20). 

Comments  from  survey  and  focus  group  participants  highlighted  several  problems  with  the  DLPT.  Some 
of  the  most  common  problems  from  survey  respondents  were  that  the  DLPT:  (1)  is  not  job-related  (24%, 
n  =  114);  (2)  lacks  a  speaking  component  (13%,  n  =  63);  (3)  has  bad  or  inappropriate  content,  format,  and 
protocol  (12%,  n  =  57);  and  (4)  is  not  a  good  measure  of  language  proficiency  (10%,  n  =  46;  Table  6,  p. 
24).  Focus  group  participants  commented  on  similar  issues  (Table  5,  p.  23). 

Overall,  findings  suggest  the  DLPT  is  an  inappropriate  test  for  SOF  due  to  its  limited  job-relatedness  and 
accuracy.  Moreover,  this  supports  USSOCOM’s  recent  policy  decision  to  move  to  a  test  that  assesses 
speaking  and  listening  proficiency,  the  two  skill  OPI. 17 

Detailed  Findings 

DLPT’s  Job-Relatedness 

SOF  operators  and  leaders  do  not  view  the  DLPT  as  job-related.  The  majority  of  both  groups  reported 
that  it  was  not  related  or  only  slightly  related  to  what  SOF  operators  do  on  the  job  (Figure  9,  p.  19),  with 
SOF  operators  (M=  2.3)  reporting  significantly  lower  levels  of  job-relatedness  than  SOF  leaders  (M  = 
2.6).  SOF  operators  (M=  2.4)  and  leaders  (M=  2.3)  also  disagreed  or  strongly  disagreed  that  the  content 
of  the  DLPT  is  related  to  what  SOF  operators  do  during  deployment  (Figure  10,  p.  19).  In  addition,  most 


15  See  Appendices  D-H  for  additional  SOF  operator  and  leader  results  by  component  and  SOF  type,  as  well  as  for 
MI  linguists  and  09L  assigned  or  attached  to  SOF  units,  students  currently  in  the  SOF  pipeline,  and  CLPMs,. 

16  See  Appendix  J  for  open-ended  comment  theme  definitions  and  example  comments. 

17  See  Oral  Proficiency  Interview  Technical  Report  #2010011006)  for  additional  findings  related  to  SOF  operator 
and  leader  reactions  to  USSOCOM’s  recent  standard  change. 
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SOF  operators  and  leaders  agree  or  strongly  agree  that  the  DLPT  does  not  cover  all  the  language  tasks 
SOF  operators  need  to  perform  when  deployed  (Figure  1 1,  p.  20).  SOF  leaders  (M=  4.1)  reported  a 
significantly  greater  level  of  agreement  with  this  item  than  did  SOF  operators  (M=  3.7). 

Figure  9.  DLPT’s  relatedness  to  what  SOF  operators  do  on  the  job 


■  Not  Related 

■  Slightly  Related 

■  Moderately  Related 

■  Related 

■  Very  Related 

0%  20%  40%  60%  80%  100% 

Note.  SOF  Leaders:  n  =  368;  SOF  Operators:  n  =  1,058.  See  Appendices  D  and  E  for  additional  results. 

Figure  10.  The  content  of  the  DLPT  is  clearly  related  to  what  SOF  operators  do  during  deployment 


SOF  Leaders 


SOF  Operators 


0%  20%  40%  60%  80%  100% 


■  Strongly  Disagree 

■  Disagree 

■  Neither  Agree  nor  Disagree 

■  Agree 

■  Strongly  Agree 


Note.  SOF  Leaders:  n  =  363;  SOF  Operators:  n  =  1,036.  See  Appendices  D  and  E  for  additional  results. 
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Figure  11.  Language  tasks  not  covered  by  the  DLPT 


■  Strongly  Disagree 

■  Disagree 

■  Neither  Agree  nor  Disagree 

■  Agree 

■  Strongly  Agree 


Note.  SOF  Leaders:  n  =  363;  SOF  Operators:  n  =  1,036.  See  Appendices  D  and  E  for  additional  results. 

In  the  previous  SOF  language  needs  assessment,  the  DLPT’s  lack  of  job-relatedness  was  identified  as  a 
major  issue  (Surface,  Poncheri,  Lemmond  &  Shetye,  2005).  Despite  developing  a  new  version,  issues  of 
the  DLPT’s  job-relatedness  still  exist.  There  were  no  significant  differences  for  operators  who  most 
recently  tested  on  the  DLPT  IV  versus  DLPT5.  Both  groups  indicated  the  DLPT  was  not  job-related  and 
did  not  cover  all  the  language  tasks  needed  while  deployed.  SOF  operators  and  leaders  still  do  not  think 
the  DLPT  is  job-related  for  most  SOF. 

In  addition  to  SOF  operators  evaluating  the  DLPT’s  job-relatedness,  Military  Intelligence  (MI)  linguists 
and  09Ls  assigned  or  attached  to  SOF  units  evaluated  how  related  the  DLPT  in  terms  of  job-relatedness. 
While  they  provided  slightly  higher  ratings  of  the  DLPT’s  job-relatedness  than  SOF  operators,  they  still 
indicated  that  the  test  neither  relates  to  what  they  do  during  deployments  nor  covers  all  the  language  tasks 
they  need  to  do  when  deployed  (Appendix  G). 

“DLPT  does  not  measure  in  any  way  the  skills  that  MI  soldiers  need  to  conduct  their 
strategic  or  tactical  mission.  ” 

MI  Linguist  or  09L  attached  to  a  SOF  unit,  1st  SFG 


DLPT’s  Accuracy 

Similar  to  their  job-relatedness  perceptions,  SOF  operators  and  leaders  do  not  believe  the  DLPT  is  an 
accurate  assessment.  Many  disagreed  or  strongly  disagreed  that  SOF  operators’  DLPT  ratings  accurately 
reflect  their  ability  to  use  the  language  while  on  the  job  (SOF  Operators:  M=  2.6;  SOF  Leaders:  M=  2.5; 
Figure  12,  p.  21).  SOF  operators  and  leaders  also  disagreed  or  strongly  disagreed  that  SOF  operators’ 
DLPT  ratings  accurately  reflect  their  ability  to  perform  job/mission-related  tasks  in  the  target  language 
(SOF  Operators:  M=  2.6;  SOF  Leaders:  M=  2.6;  Figure  13,  p.  21).  However,  34%  (n  =  351)  of  SOF 
operators  and  55%  (n  =  200)  of  SOF  leaders  agreed  or  strongly  agreed  that  SOF  operators  who  perform 
well  on  the  DLPT  are  more  likely  to  successfully  use  language  in  the  field  than  those  who  perform  poorly 
on  it  (Figure  14,  p.  22).  For  this  item,  SOF  leaders  (M=  3.4)  reported  significantly  greater  agreement 
than  did  SOF  operators  (M=  3.0). 


11/12/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011015] 


Page  20 


SOF  Language  and  Culture  Needs  Assessment  Project 


Defense  Language  Proficiency  Interview  (DLPT) 


Moreover,  some  SOF  operators  and  leaders  who  commented  specifically  on  the  DLPT5  indicated  that  it 
was  not  an  accurate  assessment  of  language  proficiency  (5.7%)  and  that  it  lacks  mission  relatedness 
(2.5%).  This  suggests  that  the  most  recent  version  of  the  DLPT  still  lacks  fundamental  components 
needed  by  the  SOF  community. 

Figure  12.  SOF  operators’  DLPT  ratings  accurately  reflect  their  ability  to  use  the  language  on  the  job 


■  Strongly  Disagree 

■  Disagree 

■  Neither  Agree  nor  Disagree 

■  Agree 

■  Strongly  Agree 

0%  20%  40%  60%  80%  100% 

Note.  SOF  Leaders:  n  =  362;  SOF  Operators:  n  =  1,037.  See  Appendices  D  and  E  for  additional  results. 

Figure  13.  SOF  operators’  DLPT  ratings  accurately  reflect  their  ability  to  do  job/mission-related  tasks  in 
the  target  language 


SOF  Leaders 


SOF  Operators 


~\ - 1 - 1 - 1 - 1 - 1 

0%  20%  40%  60%  80%  100% 


■  Strongly  Disagree 

■  Disagree 

■  Neither  Agree  nor  Disagree 

■  Agree 

■  Strongly  Agree 


Note.  SOF  Leaders:  n  =  361;  SOF  Operators:  n  =  1,036.  See  Appendices  D  and  E  for  additional  results. 


11/12/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011015] 


Page  21 


SOF  Language  and  Culture  Needs  Assessment  Project 


Defense  Language  Proficiency  Interview  (DLPT) 


Figure  14.  DLPT  related  to  SOF  operators  field  performance 


SOF  Leaders 


SOF  Operators 


T 


1 


0%  20%  40%  60%  80%  100% 


■  Strongly  Disagree 

■  Disagree 

Neither  Agree  nor  Disagree 

■  Agree 

■  Strongly  Agree 


Note.  SOF  Leaders:  n  =  368;  SOF  Operators:  n  =  1,058.  See  Appendices  D  and  E  for  additional  results. 


Similar  to  job-relatedness,  perceptions  of  DLPT  accuracy  have  not  improved  with  the  DLPT5.  In  fact, 
perceptions  have  gotten  more  negative.  SOF  operators,  regardless  of  version  (DLPT  IV  or  DLPT5), 
indicated  that  the  DLPT  is  not  an  accurate  assessment;  however,  SOF  operators  who  last  tested  on  the 
DLPT  IV  viewed  the  DLPT  as  slightly  more  accurate  than  those  who  last  took  the  DLPT5  (Table  4,  p. 
22). 


Table  4.  SOF  operators’  DLPT  accuracy  perceptions  by  most  recent  version 


Last  took  DLPT  IV 

Last  took  DLPT5 

N 

Mean 

N 

Mean 

My  DLPT  ratings  accurately  reflect  my  ability  to  use 
language  while  on  the  job. 

211 

2.7 

452 

2.5 

My  DLPT  ratings  are  an  accurate  reflection  of  my  ability 
to  perform  job/mission-related  tasks  in  the  target  language. 

211 

2.8 

450 

2.5 

SOF  operators  who  perform  well  on  the  DLPT  are  more 
likely  to  successfully  use  language  in  the  field  than  those 
who  perform  poorly  on  the  DLPT. 

211 

3.1 

449 

2.9 

Note.  The  scale  for  the  first  item  is:  1  =  Not  Related,  2  =  Slightly  Related,  3  =  Moderately  Related,  4  =  Related,  5  =  Very  Related.  The  scale  for 
the  last  two  items  is:  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


As  with  their  perceptions  of  the  DLPT’s  job-relatedness,  MI  linguists  and  09Ls  assigned  or  attached  to 
SOF  units  reported  slightly  higher  ratings  than  SOF  operators  concerning  the  DLPT’s  accuracy. 
However,  many  still  strongly  disagreed  or  disagreed  that  it  accurately  reflected  their  ability  to  use  their 
target  language  on  the  job  (Appendix  G). 
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' While  DLPT5  is  extremely  challenging,  the  Ph.Ds  who  designed  it  need  to  tailor  it  to 
those  who  are  taking  it.  A  Mobile  Training  Team  came  to  us  at  Fort  Meade  and 
explained  that  the  designers  suggest  getting  GRE  level  vocabulary  preparatory  materials 
(in  English)  to  prepare  for  the  test.  This  is  ridiculous.  Many  of  those  who  are  going 
through  DLI  are  right  out  of  high  school;  at  best,  they  may  have  taken  the  SATs.  Either 
they  need  to  make  more  levels  of  test  available  (maybe  lower,  intermediate,  upper)  or 
make  it  more  reasonable  for  a  wider  audience.  ” 

MI  Linguist  or  09L,  Theater  Special  Operations  Command  (TSOC) 

Survey  and  Focus  Group  Comments 

SOF  operators  and  leaders  provided  additional  feedback  on  the  DLPT  during  focus  groups  (Table  5,  p. 

23)  and  through  survey  comments  (Table  6,  p.  24). 18  Overall,  the  majority  of  survey  comments  (79%) 
were  negative  in  nature  (Table  6,  p.  24),  while  7%  were  positive. 19  The  most  frequently  occurring  themes 
focused  on  negative  reactions  to  the  DLPT:  not  being  job  relevant;  not  being  an  accurate  assessment  of 
language  proficiency;  not  including  a  speaking  component;  including  bad  or  inappropriate  test  content  or 
format;  and  being  too  difficult.  Example  comments  from  survey  respondents  and  focus  group  participants 
are  provided  below  common  themes. 

Table  5.  Focus  group  participants’  DLPT  feedback 


Theme 

Number  of  Segments 

DLPT  (General) 

DLPT5 

DLPT  is  a  poor  indicator  of  proficiency 

34 

13 

DLPT  content  is  unrelated  to  mission/job/military 

38 

8 

DLPT  is  too  difficult 

13 

13 

Other  negative  DLPT  comments 

16 

3 

Other  positive  DLPT  comments 

7 

1 

Note.  Focus  group  segments  coded  “DLPT  (General)”  either  mentioned  other  versions  of  the  DLPT  than  the  DLPT5  or  did  not  specify  a  version. 
Focus  group  segments  coded  “DLPT5”  specifically  mentioned  the  DLPT5. 


18  Focus  group  segments  and  survey  comments  were  separated  according  to  whether  the  DLPT5  was  specifically 
mentioned  (Appendix  I,  Table  1,  p.  56)  or  whether  it  was  a  generic  comment  about  the  DLPT  (Appendix  I,  Table  2, 
P-  57). 

19  Percentages  may  not  add  to  100%  as  18%  of  comments  were  coded  as  N/A  and  6%  of  comments  were  coded  as 
both  positive  and  negative  toward  the  DLPT. 
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Table  6.  Survey  respondents’  DLPT  feedback 


Theme 

Percentage  of  Total  Comments 

SOF 

Operators 

SOF 

Leaders 

Combined 

Negative  Comments 

DLPT  content  is  unrelated  to  mission/job/military 

23.4% 

25.4% 

24.2% 

DLPT  is  not  an  accurate/valid  assessment 

15.6% 

19.0% 

17.0% 

Needs  to  include  a  speaking  component 

12.0% 

15.3% 

13.4% 

Test  content/format/protocol  is  bad/not  appropriate 

9.9% 

15.3% 

12.1% 

DLPT  is  too  difficult 

7.8% 

10.6% 

8.9% 

Leadership/policy  problems  related  to  DLPT  testing 

6.0% 

12.2% 

8.5% 

Needs  to  cover  more  military-specific  topics 

7.8% 

8.5% 

8.1% 

General  negative 

3.5% 

10.6% 

6.4% 

Training  does  not  match  what  is  tested  on  the  DLPT 

4.3% 

3.7% 

4.0% 

Not  able  to  prepare  for  the  test 

1.1% 

2.1% 

1.5% 

Positive  Comment 

DLPT  is  an  accurate/valid  assessment 

3.2% 

6.3% 

4.5% 

General  positive 

0.7% 

2.1% 

1.3% 

Test  content/format/protocol  is  good/appropriate 

0.7% 

1.6% 

1.1% 

Training  matches  what  is  tested  on  the  DLPT 

0.7% 

0.5% 

0.6% 

DLPT  content  is  related  to  mission/job/military 

- 

0.5% 

0.2% 

Total  Comments 

282 

189 

471 

Note.  See  Appendix  J  for  definitions  and  example  comments  for  each  theme.  Comments  for  “None/NA/not  relevant  response”  were  not  included 
in  this  table.  Comments  coded  “DLPT  (General)”  either  mentioned  other  versions  of  the  DLPT  than  the  DLPT5  or  did  not  specify  a  version. 
Comments  coded  “DLPT5”  mentioned  the  DLPT5.  Percents  may  not  add  to  100%  since  some  respondents  may  have  made  both  positive  and 
negative  statements  in  a  single  comment. 


One  of  the  most  common  themes  was  that  the  DLPT  is  unrelated  to  what  SOF  operators  do  while 
deployed: 


“regarding  the  German  DLPT \  the  questions/answers  have  almost  nothing  to  do  with  the 
role  language  proficiency  plays  in  our  SOF  operations/training 

SOF  Leader,  10th  SFG 

“it  is  not  what  SOF  soldiers  need  to  be  effective  on  the  modern  battle  field,  reading  and 
writing  have  nothing  to  do  with  what  I  have  done  on  4  deployments  ” 

SOF  Operator,  5th  SFG 
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Survey  respondents  and  focus  group  participants  also  mentioned  that  they  do  not  believe  the  DLPT  is  an 
accurate  assessment  of  language  proficiency: 

“I  do  not  feel  the  DLPT  V  is  an  accurate  assessment  of  the  language  ability  of  the  test 
taker,  it  seems  to  test  subjectively  instead  of  objectively.  /  Questions  ask  about  "how  does 
the  author/ speaker  feel "  instead  of  what  the  author/speaker  is  writing  /  Answers  to  the 
questions  seem  to  be  ambiguous  as  there  appear  to  be  more  than  one  correct  option 
listed  as  choices  ” 

SOF  Operator,  20th  SFG 

The  need  for  language  testing  to  include  a  speaking  component  was  also  discussed  during  the  focus 
groups  and  from  survey  respondents.  Findings  establishing  a  need  for  speaking  in  language  testing  were 
also  found  when  assessing  reactions  to  USSOCOM’s  recent  standard  change. 20  From  these  comments,  it 
seems  that  part  of  the  reason  they  do  not  believe  the  DLPT  is  related  to  SOF  operators’  job  is  that  they 
primarily  use  speaking  (or  speaking  and  listening)  skills  while  deployed,  but  the  DLPT  only  tests  reading 
and  listening: 

(L  think  another  thing  with  the  DLPT  is  that  it  emphasizes  reading  a  lot,  and  I  think  if 
you  ’re  being  realistic  about  the  way  we  ’re  going  to  use  language,  it  should  just  be 
listening  and  speaking.  ” 

1st  SFG  Focus  Group 

Survey  participants  provided  negative  comments  about  the  DLPT’s  content,  format,  and  testing  protocol. 
Specifically,  some  survey  participants  mentioned  problems  associated  with  the  use  of  English  on  the  test, 
the  test  length,  and  the  lack  of  a  non-verbal  communication  section: 

“It's  a  6-hour  test  for  Arabic.  Does  it  really  need  to  be  that  long?  [...JIs  there  a  way  to 
shorten  the  test  back  to  the  old  2-hour  format?  ” 

SOF  Leader, 

United  States  Army  Special  Operations  Command  Headquarters  (USASOC  HQ) 

SOF  operators  and  leaders  perceive  the  DLPT,  particularly  the  DLPT5,  as  too  difficult.  Survey 
respondents  and  focus  group  participants  mentioned  that  the  test  assessed  language  proficiency  at  a  level 
too  high  for  most  SOF  personnel: 

‘‘The  DLPT  4  was  a  fair  test  or  basic  language,  [...] .  The  DLPT 5  was  like  taking  the  SAT 
in  Arabic.  ” 

SOF  Leader,  3rd  SFG 


20  Oral  Proficiency  Interview  (OPI;  Technical  Report  #2010011016) 
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“DLPT  seems  to  be  set  at  a  college  level  discussion.  A  large  portion  focused  on  politics. 

The  average  person  dealt  with  on  deployments  are  not  at  that  level  and  rarely  discuss 
global  warming.  ” 

SOF  Operator,  USSOCOM  Operational  Unit 

While  most  provided  negative  feedback  on  the  DLPT,  some  participants  made  favorable  comments 
during  the  focus  groups  and  on  the  survey  (Table  5,  p.  23).  However,  they  often  paired  these  positive 
aspects  of  the  DLPT  with  its  limitations: 

“The  DLPT  accurately  measures  an  operators  overall  ability  in  a  given  language  but  it 
does  not  accurately  measure  the  operator's  ability  to  use  the  language  to  accomplish  a 
given  military  mission  down  range” 

SOF  Leader,  3rd  SFG 

“I  think  the  reading  needs  to  stay  on,  as  a  team  sergeant  or  as  a  leader,  or  as  whatever, 

Tm  going  to  be  on  the  internet  and  Tm  going  to  be  getting  stuff  from  the  Embassy  here; 
locally  Tm  going  to  be  getting  stuff  in  the  newspaper  or  something  like  that.  It  could  be  a 
threat  to  the  rest  of  my  guys,  and  Tm  going  to  be  able  to  read  that.  ” 

AFSOC  Focus  Group 
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SECTION  V:  CONCLUSIONS 

This  report  examines  perceptions  of  the  DLPT  in  the  Special  Operations  Forces  (SOF)  community, 
specifically  SOF  leader  support  of  DLPT  language  testing,  SOF  operator  DLPT  testing  experiences,  and 
SOF  operator  and  leader  views  of  the  DLPT.  While  USSOCOM  recently  changed  its  test  of  record  for 
language  proficiency  to  the  OPI,  some  components  and  Services  still  test  on  the  DLPT.  Responses 
indicate  that  (1)  SOF  leaders  do  not  fully  support  the  use  of  the  DLPT  for  language  proficiency  testing  in 
the  SOF  community,  (2)  almost  half  of  SOF  operators  reported  experiencing  scheduling  and  technical 
problems  during  DLPT  testing,  and  (3)  SOF  operators  and  leaders  indicated  that  the  DLPT  does  not 
assess  the  language  proficiency  skills  needed  for  missions. 

SOF  Leader  Support  for  DLPT  Testing 

As  part  of  their  leadership  role,  SOF  leaders  should  encourage  and  monitor  SOF  operator  status  on 
meeting  USSOCOM  language -testing  requirements.  While  USSOCOM  recently  changed  its  test  of 
record  for  language  proficiency  to  the  OPI,  at  the  time  of  this  survey,  many  SOF  leaders  indicated  that 
their  units  required  SOF  operators  to  test  on  the  DLPT  to  fulfill  language  testing  requirements.  In  the 
future,  many  SOF  operators  will  continue  to  test  on  the  DLPT  to  receive  FLPB. 

SOF  leaders  should  pay  attention  to  SOF  operators’  DLPT  ratings  and  encourage  them  to  do  well  on  the 
test.  However,  many  SOF  leaders  reported  not  paying  attention  to  DLPT  scores,  not  believing  DLPT 
ratings  are  important,  and  not  encouraging  SOF  operators  to  stay  up-to-date  on  language  testing  or  do 
well  on  the  DLPT.  Moreover,  of  SOF  leaders  in  units  that  require  personnel  to  take  the  DLPT  and  who 
could  comment  on  DLPT  testing  (n  =  371),  51%  (n  =  183)  indicated  their  units  fell  below  USSOCOM’s 
requirement  of  more  than  80%  of  their  personnel  meeting  language  testing  requirements.  Therefore, 
approximately  half  of  SOF  units  did  not  meet  USSOCOM’s  standard  and  should  be  following  mandated 
language  training  programs.21 

SOF  Operator  Experiences  with  the  DLPT 

During  their  most  recent  DLPT,  44%  {n  =  468)  of  SOF  operators  reported  experiencing  problems.  The 
most  commonly  reported  problems  included  computer/technical  issues  (24%,  n  =  257)  and  test  scheduling 
delays  (22%,  n  =  235).  Computer/technical  issues  SOF  operators  experience  should  be  communicated  to 
the  Defense  Language  Institute  (DLI)  testing  authorities  since  problems  may  have  a  negative  impact  on 
performance.  Specifically,  technical  issues  can  impact  the  reliability  and  validity  of  test  scores  and,  in 
extreme  cases,  the  validity  as  well.  Test  scheduling  delays  should  addressed  since  may  be  affect  SOF 
operators’  abilities  to  stay  up-to-date  on  their  language  testing  requirement  if  they  are  required  to  take  the 
DLPT.  Although  outside  the  direct  control  of  SOF,  SOF  leaders  can  raise  these  issues  with  those  who 
have  propensity  and  oversight  for  testing  and  test  delivery 


21  However,  these  are  only  SOF  leader  estimates  rather  than  actual  counts.  If  there  is  a  discrepancy  between  these 
estimates  and  actual  counts,  the  discrepancy  suggests  that  SOF  leaders  are  not  paying  close  attention  to  SOF 
operator  language  test  scores. 
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Perceptions  of  DLPT  Accuracy  and  Job-relatedness 

Findings  support  USSOCOM’s  recent  change  from  emphasizing  reading  and  listening  skills  (as  measured 
by  the  DLPT)  to  the  emphasizing  speaking  and  listening  skills  (as  measured  by  the  OPI).  SOF  operators 
and  leaders  indicated  that  the  DLPT  neither  relates  to  their  jobs  nor  accurately  assesses  language 
proficiency.  They  also  indicated  the  DLPT  does  not  cover  all  the  language  tasks  needed  when  deployed. 
Specifically,  most  SOF  operators  and  leaders  reported  that  the  DLPT  was  not  related  or  only  slightly 
related  to  what  SOF  operators  do  on  the  job  and  agreed  or  strongly  agreed  that  the  DLPT  does  not  cover 
all  the  language  tasks  SOF  operators  need  to  perform  when  deployed.  Therefore,  the  move  away  from 
testing  on  the  DLPT  to  OPI  testing  supports  USSOCOM’s  goals  of  emphasizing  mission-related  language 
skills.22  Additionally,  recent  research  sponsored  by  SOFLO  indicates  the  DLPT  listening  is  not  an 
effective  proxy  for  speaking,  which  is  the  desired  skill  for  most  SOF.23  Therefore,  this  further  supports 
the  transition  to  OPI  testing. 

Additionally,  findings  mirror  those  found  during  the  2004  SOF  Language  Transformation  Needs 
Assessment  (Surface,  Poncheri,  Lemmond  &  Shetye,  2005).  When  responding  to  that  survey,  SOF 
operators  and  leaders  indicated  that  the  DLPT  did  not  accurately  measure  language  proficiency  or  how 
they  use  language  on  missions.  Moreover,  perceptions  have  gotten  more  negative  as  SOF  operators  who 
last  tested  on  the  DLPT  IV  viewed  the  DLPT  as  slightly  more  accurate  than  those  who  last  took  the 
DLPT5.  As  in  2004,  the  findings  indicate  that  the  DLPT  does  not  meet  the  testing  needs  of  the  SOF 
community  and  does  not  facilitate  the  alignment  of  capability  with  mission  requirements.  However, 
many  in  SOF  will  still  take  the  DLPT  to  qualify  for  Foreign  Language  Proficiency  Bonus  (FLPB), 
making  these  findings  problematic  for  the  SOF  community  and  the  goal  of  developing  mission-relevant 
foreign  language  capability.  USSOCOM  and  SOF  component  leadership  should  take  steps  to  address 
these  concerns  to  better  align  foreign  language  proficiency  testing  with  the  needs  of  the  SOF  community. 


22  See  Oral  Proficiency  Interview  (OPI;  Technical  Report  #201001 1016)  for  more  information  on  SOF  operator  and 
leader  perspectives  on  the  standard  change. 

23  Using  the  DLPT  as  a  Proxy  for  the  OPI:  Are  Reading  and  Non-Participatory  Listening  Scores  a  Subsitute  for 
Direct  Assessment  of  Speaking  Proficiency?  (Technical  Report  #2010010624) 
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ABOUT  SWA  CONSULTING  INC. 

SWA  Consulting  Inc.  (formerly  Surface,  Ward,  and  Associates)  provides  analytics  and  evidence-based 
solutions  for  clients  using  the  principles  and  methods  of  industrial/organizational  (I/O)  psychology.  Since 
1997,  SWA  has  advised  and  assisted  corporate,  non-profit  and  governmental  clients  on: 

•  Training  and  development 

•  Performance  measurement  and  management 

•  Organizational  effectiveness 

•  Test  development  and  validation 

•  Program/training  evaluation 

•  Work/job  analysis 

•  Needs  assessment 

•  Selection  system  design 

•  Study  and  analysis  related  to  human  capital  issues 

•  Metric  development  and  data  collection 

•  Advanced  data  analysis 

One  specific  practice  area  is  analytics,  research,  and  consulting  on  foreign  language  and  culture  in  work 
contexts.  In  this  area,  SWA  has  conducted  numerous  projects,  including  language  assessment  validation 
and  psychometric  research;  evaluations  of  language  training,  training  tools,  and  job  aids;  language  and 
culture  focused  needs  assessments  and  job  analysis;  and  advanced  analysis  of  language  research  data. 

Based  in  Raleigh,  NC,  and  led  by  Drs.  Eric  A.  Surface  and  Stephen  J.  Ward,  SWA  now  employs  close  to 
twenty  I/O  professionals  at  the  masters  and  PhD  levels.  SWA  professionals  are  committed  to  providing 
clients  the  best  data  and  analysis  upon  which  to  make  evidence-based  decisions.  Taking  a  scientist- 
practitioner  perspective,  SWA  professionals  conduct  model-based,  evidence-driven  research  and 
consulting  to  provide  the  best  answers  and  solutions  to  enhance  our  clients’  mission  and  business 
objectives.  SWA  has  competencies  in  measurement,  data  collection,  analytics,  data  modeling,  systematic 
reviews,  validation,  and  evaluation. 

For  more  information  about  SWA,  our  projects,  and  our  capabilities,  please  visit  our  website  (www.swa- 
consulting.com)  or  contact  Dr.  Eric  A.  Surface  (esurface@swa-consulting. com)  or  Dr.  Stephen  J.  Ward 
(sward@swa-consulting.com). 

The  SWA  Consulting  Inc.  team  members  contributed  to  this  report  (listed  in  alphabetical  order): 

Mr.  Kartik  S.  Bhavsar  Ms.  Jenna  Hartinger 

Ms.  Sarah  C.  Bienkowski  Dr.  Eric  A.  Surface 

Mr.  Milton  Cahoon  Dr.  Stephen  J.  Ward 

Dr.  Reanna  Poncheri  Harman  Ms.  Sheila  L.  Wilcox 
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APPENDIX  A:  ABOUT  THE  LCNA  PROJECT 

In  2003-2004,  the  Special  Operations  Forces  Language  Office  (SOFLO)  sponsored  the  SOF  Language 
Transformation  Strategy  Needs  Assessment  Project  to  inform  the  development  of  a  language 
transformation  strategy  in  response  to  a  GAO  report  (2003).  This  SOF  Language  Transformation  Strategy 
Needs  Assessment  Project  collected  current-state  information  about  language  usage,  proficiency,  training, 
and  policy  issues  (e.g.,  Foreign  Language  Proficiency  Pay,  FLPP)  from  SOF  personnel,  SOF  unit  leaders, 
and  other  personnel  involved  in  SOF  language.  The  project  used  multiple  data  collection  methods  and 
provided  the  SOFLO  with  valid  data  to  develop  a  comprehensive  language  transformation  strategy  and 
advocate  for  the  SOF  perspective  on  language  issues  within  the  DoD  community. 

In  a  continuing  effort  to  update  knowledge  of  language  and  culture  needs  while  informing  strategic  plan 
development,  the  SOFLO  commissioned  the  2009  SOF  Language  and  Culture  Needs  Assessment  Project 
(LCNA)  to  reassess  the  language  and  culture  landscape  across  the  United  States  Special  Operations 
Command  (USSOCOM)  and  develop  a  strategy  for  the  next  five  years.  Data  were  collected  between 
March  and  November,  2009  from  personnel  in  the  SOF  community,  including  SOF  operators  and  leaders. 
Twenty-three  focus  groups  were  conducted  between  March  and  June,  2009.  A  comprehensive,  web-based 
survey  for  SOF  operators  and  leaders  was  launched  on  26  October  and  closed  on  24  November,  2009. 

This  project’s  findings  will  be  disseminated  through  reports  and  briefings  (see  Appendix  A,  Figure  1). 
Two  foundational  reports  document  the  methodology  and  participants  associated  with  this  project.  The 
remaining  reports  are  organized  in  three  tiers.  Twenty-five  Tier  /reports  focus  on  specific,  limited  issues 
(e.g.,  Inside  AOR  Use  of  Language).  Tier  II  reports  integrate  and  present  the  most  important  findings 
across  related  Tier /reports  (e.g.,  Use  of  Language  and  Culture  on  Deployment)  while  including 
additional  data  and  analysis  on  the  topic.  Most,  but  not  all,  Tier  /reports  will  roll  into  Tier  II  reports.  One 
Tier  III  report  presents  the  most  important  findings,  implications,  and  recommendations  across  all  topics 
explored  in  this  project.  The  remaining  Tier  III  reports  present  findings  for  specific  SOF  organizations 
[e.g.,  Air  Force  Special  Operations  Command  (AFSOC),  Special  Forces  (SF)  Command].  All  Tier  III 
reports  are  associated  with  a  briefing.  Report  topics  are  determined  by  the  SOFLO  and  subject  to  change. 

In  June,  2009,  the  GAO  reported  that  the  Department  of  Defense  is  making  progress  toward  transforming 
language  and  regional  proficiency  capabilities  but  still  does  not  have  a  strategic  plan  in  place  to  continue 
development  that  includes  actionable  goals  and  objectives.  The  findings  from  this  study  can  be  used  by 
the  SOFLO  and  leaders  at  USSOCOM  to  continue  strategic  planning  and  development  in  this  area. 

This  project  design,  logistics,  data  collection,  initial  analysis  and  first  eight  reports  of  this  project  were 
conducted  by  SWA  Consulting  Inc.  (SWA)  under  a  subcontract  with  SRC  (SR20080668  (K142);  Prime  # 
N65236-08-D-6805).  The  additional  reports  are  funded  under  a  separate  contracting  vehicle  with  Gemini 
Industries  Inc.  [GEM02-ALMBOS-0018  (10210SWA-1);  Prime  #  USZA22-02-D-0015].  For  questions  or 
more  information  about  the  SOFLO  and  this  project,  please  contact  Mr.  Jack  Donnelly 
(i ohn. donnellv@socom.  mil).  For  specific  questions  related  to  data  collection  or  reports  associated  with 
this  project,  please  contact  Dr.  Eric  A.  Surface  ( esurface@s  wa-consulting. com)  or  Dr.  Reanna  Poncheri 
Harman  (rpharman@swa-consulting.com)  with  SWA  Consulting  Inc. 
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Appendix  A,  Figure  1.  Report  Overview 


Foundation  Reports 


1.  Methodology  Report 

2.  Participation  Report 


Tier  I  Reports  First  Contract 


3.  Reactions  to  Admiral  Olson's  Memo 

4.  Training  Emphasis:  Language  and  Culture 

5.  Command  Support:  Grading  the  Chain  of 
Command 

6.  SOFLO  Support 

7.  Inside/Outside  AOR  Use  of  Cultural  Knowledge 

8.  Language  Composition  of  SOF  Tactical  Elements 


Tier  I  Reports  Second  Contract 


9.  Inside  AOR  Use  of  Language 

10.  Outside  AOR  Use  of  Language 

11.  Mission-Specific  Use  of  Interpreters 

12.  General  Use  of  Interpreters 

13.  09L  Use  in  the  Special  Operations  Forces 
Community 

14.  DLPT 

15.  OPI 

16.  DLAB:  Perspectives  from  the  Field 

17.  Initial  Acquisition  Training 

18.  Sustainment/Enhancement  Training 

19.  Culture  Awareness  and  Knowledge  Training 

20.  Immersion  Training 

21.  Language  Resources  &  Self-Study 

22.  Foreign  Language  Proficiency  Bonus 

23.  Non-monetary  Incentives 

24.  Considering  Language  in  the  Promotion  Process 

25.  Barriers  to  Language  Acquisition  and  Maintenance 

26.  Force  Motivation  for  Language 

27.  Leader  Perspectives  on  Language  Issues 

28.  Leader  Perspectives  on  Language  Resources 

29.  CLPM  Perspectives 


Tier  II  Reports  Second  Contract 


30.  Use  of  Language  and  Culture  on  Deployment 

31.  Use  of  Interpreters 

32.  Tactical  Element  Composition  and  Capability 

33.  Testing/Metrics 

34.  Current  State  of  Language  and  Culture  Training 

35.  Language  Training  Guidance 

36.  Culture  Training  Guidance 

37.  Incentives/Barriers 

i _ 


Tier  III  Reports  Second  Contract 


38.  Overall  Picture:  Conclusions  and 
Recommendations 

39.  AFSOC 

40.  MARSOC 

41.  WARCOM 

42.  SF  Command 

43.  CA 

44.  MISG 

45.  Seminar  Briefing(s) 


Note :  Foundation  reports  are  referenced  by  every  other  report.  Colors  represent  Tier  I  reports  that  roll  (integrate)  into  an  associated  Tier  II  report.  Reports  in  black  are  final  reports  on  the  topic 
but  may  be  cited  by  other  reports.  Tier  //reports  roll  into  the  Tier  ///reports.  All  Tier  III  reports  include  an  associated  briefing. 
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APPENDIX  B:  METHODOLOGY 

Participants 

Focus  Group 

SOF  personnel  ( n  =  126)  participated  in  23  focus  groups  conducted  across  the  SOF  community,  including 
Air  Force  Special  Operations  Command  (AFSOC),  Marine  Corps  Forces  Special  Operations  Command 
(MARSOC),  Naval  Special  Warfare  Command  (WARCOM),  and  United  States  Army  Special  Operations 
Command  (USASOC). 24  This  report  includes  comments  and  content  (i.e.,  theme)  totals  from  focus 
group  participants  regarding  the  DLPT. 

Survey 

Survey  respondents  received  the  SOF  operator  version  of  the  DLPT  items  if  they  indicated  one  of  the 
following  roles: 

•  SOF  Operator 

•  SOF  Operator  assigned  to  other  duty 

•  Currently  in  the  training  pipeline  for  SOF 

•  MI  Linguist  or  09L  assigned  or  attached  to  a  SOF  unit 

•  SOF  Retirees 

Respondents  received  the  SOF  leader  version  of  the  DLPT  and  language  testing  items  if  they  designated 
one  of  the  following  roles: 

•  SOF  Unit  Commanders  and  Unit  Leadership  of  03  Commands  or  higher,  including  Staff, 
Support,  and  Specialists 

•  Command  Language  Program  Manager  or  Component  Language  Program  Manager  (CLPM) 

•  Language  Office  Personnel 

•  Instructor 

Throughout  this  report,  “SOF  operator”  refers  to  respondents  who  indicated  they  were  a  SOF  Operator  or 
a  SOF  Operator  assigned  to  other  duty.  The  “SOF  leader”  responses  throughout  this  report  only  include 
respondents  who  indicated  they  were  SOF  Unit  Commanders  and  Unit  Leadership  of  03  Commands  or 
higher. 

For  comparisons  to  SOF  operators,  Appendix  E  provides  responses  from  “MI  Linguist  or  09L  assigned  or 
attached  to  SOF  unit”  and  Appendix  F  provides  responses  from  students  currently  in  a  SOF  training 
pipeline.  For  comparisons  to  SOF  leaders,  Appendices  H  provide  CLPM  responses. 


24  See  Participation  Report  (Technical  Report  #201001 1003)  for  participant  details.  See  Methodology  Report 
(Technical  Report  #201001 1002)  for  the  focus  group  interview  guide. 
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This  report  excluded  respondents  who  indicated  they  were  a  SOF  retiree,  Language  Office  Personnel,  or 
Instructor. 

SOF  Type  Classification 

Appendices  G  and  H  present  SOF  operators  and  leaders  assigned  to  US  ASOC  according  to  their  SOF 
type  [i.e.,  Civil  Affairs  (CA),  Military  Intelligence  Support  Group  (MISG),  and  Special  Forces  (SF)]. 

This  report  presents  USASOC  respondents  classified  into  SOF  types  based  on  the  following  criteria25: 

•  Assigned  USASOC  unit  (e.g.,  1st  SFG  classified  as  SF,  4th  POG  as  MISG,  95th  CA  Bde  as  CA) 

•  Reported  MOS  (e.g.,  18  series  were  classified  as  SF,  37  series  as  MISG,  38  series  as  CA) 

Measures 

SOF  Operators 

Respondents  received  12  closed-ended  and  2  open-ended  items  related  to  the  DLPT  if  they  responded  in 
the  affirmative  to  the  following  item: 

•  Have  you  ever  taken  the  Defense  Language  Proficiency  Test  (DLPT)? 

Respondents  who  indicated  “Yes”  (92.9%,  n  =  1,086)  received  the  DLPT  items.  Respondents  who 
indicated  “No”  (7. 1%,  n  =  83)  were  branched  to  the  next  survey  section.  The  DLPT  items  asked  if  they 
were  currently  required  to  test  in  an  official  or  required  language,  when  they  last  took  the  DLPT,  and 
which  version  they  last  took.  In  addition,  the  survey  asked  about  any  problems  experienced  during  their 
last  testing  event,  as  well  as  their  perceptions  of  the  DLPT’s  job-relatedness  and  accuracy  in  assessing 
their  language  skills. 

SOF  Leaders 

SOF  leaders  received  items  related  to  the  DLPT  and  language  testing  if  they  responded  in  the  affirmative 
to  the  following  item: 

•  Are  personnel  in  your  unit  required  to  take  the  Defense  Language  Proficiency  Test  (DLPT)?  (If 
yes,  are  you  in  a  position  to  comment  on  this?) 

Respondents  that  indicated  “Yes,  and  I  am  in  a  position  to  comment  on  the  DLPT”  (48.8%,  n  =  371) 
received  the  DLPT  and  language  testing  items.  Respondents  that  indicated  “Yes,  but  I  am  NOT  in  a 
position  to  comment  on  the  DLPT”  (40.5%,  n  =  308)  or  “No,  personnel  in  my  unit  are  not  required  to  take 
the  DLPT”  (10.7%,  n  =  81)  were  branched  to  the  next  survey  section.  The  DLPT  section  included  nine 


25  For  further  details  on  participation  and  attrition  rates,  please  refer  to  the  Participation  Report  (Technical  Report 
#2010011003). 
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closed-ended  items  and  one  open-ended  item  asking  about  their  experiences  with  and  perceptions  of  SOF 
operator  DLPT  testing.  SOF  leaders  also  received  two  closed-ended  items  asking  about  their  experiences 
with  official  language  testing  in  their  units. 

Analyses 

Analysis  of  all  closed-ended  items  consisted  of  a  combination  of  descriptive  and  inferential  statistics. 
Appendices  C  -  H  present  frequencies  and  average  (i.e.,  mean)  responses  for  several  items.  Analysts  used 
inferential  statistics  (e.g.,  analysis  of  variance,  t- tests)  to  determine  whether  significant  observed 
differences  existed  across  groups  of  participants  (e.g.,  SOF  operators  v.  SOF  leaders)  that  may  suggest 
differences  in  the  broader  population  of  interest. 

For  qualitative  survey  data  (i.e.,  open-ended  comments),  raters  created  a  content  code  (i.e.,  theme)  list 
based  on  available  responses  for  each  item.  A  primary  rater  then  coded  each  response  and  a  secondary 
rater  coded  30%  of  the  responses.  Raters  determined  the  consistency  of  codes  and  discussed  any 
disagreements  to  consensus.  This  report  presents  frequencies  of  occurrence  for  each  theme  throughout. 
Focus  group  data  coding  used  a  similar  process.26  Two  different  coders  from  those  who  coded  the  survey 
comments  rated  the  content  of  each  focus  group  segment.  Totals  for  themes  and  verbatim  comments 
related  to  DLPT  testing  and/or  language  testing  requirements  are  provided  throughout  the  report. 


26  For  further  details  on  these  methods,  please  refer  to  the  Methodology  Report  (Technical  Report  #201001 1002). 
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APPENDIX  C:  LANGUAGE  TESTING  REQUIREMENT 

Appendix  C,  Table  1.  SOF  Leaders:  Encouragement  of  SOF  operators  to  stay  up-to-date  on  the  language  testing  requirement 


Group 

n 

M 

Never  Seldom 

Sometimes 

Often 

Very  Often 

Overall 

361 

3.95 

3% 

7% 

15% 

40% 

34% 

USSOCOM  HQ 

41 

3.61 

5% 

15% 

20% 

37% 

24% 

JSOC 

1 

4.00 

0% 

0%| 

0% 

100% 

0% 

TSOC 

14 

3.93 

0% 

7% 

29% 

29% 

36% 

Deployed  SO  Unit 

27 

4.30 

0% 

0% 

15% 

41% 

44% 

AFSOC 

2 

3.50 

0% 

0% 

50% 

50% 

0% 

MARSOC 

3 

4.00 

0% 

0% 

0% 

100% 

0% 

USASOC 

266 

3.98 

3% 

7% 

14% 

40% 

36% 

CA 

31 

3.90 

3% 

6% 

13% 

52% 

26% 

MISG 

55 

4.15 

2% 

2% 

11% 

51% 

35% 

SF 

153 

4.03 

4% 

7% 

13% 

36% 

41% 

Note.  1  =  Never,  2  =  Seldom,  3 
leaders  responded  to  this  item. 

=  Sometimes ,  4 

=  Often,  5  =  Very  Often.  Army  SOF  types  did  not  significantly  differ.  Differing  sample  sizes  prevented  SOF  component  comparisons.  No  WARCOM 

Appendix  C,  Table  2. 

CLPMs:  Encouragement  of  SOF  operators  to  stay  up-to-date  on 

the  language  testing  requirement 

Group  n  M  Never 

Seldom 

Sometimes 

Often 

Very  Often 

CLPMs  9  4.67  0%! 

0%l 

0%l 

33% 

67%  ■ 

Note.  1  =  Never,  2  =  Seldom,  3  =  Sometimes,  4  =  Often,  5  =  Very  Often. 
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Appendix  C,  Table  3.  SOF  Leaders:  Estimated  percentage  of  SOF  operators  up-to-date  on  language  testing  requirement 


Group 

n 

M 

0-20%  21-40% 

41-60% 

61-80%  81-100% 

Overall 

356 

1A1 

8% 

10% 

10% 

23% 

49% 

USSOCOM  HQ 

39 

7.49 

15% 

5%| 

8% 

15% 

56% 

JSOC 

1 

4.00 

0% 

100% 

0% 

0% 

0% 

TSOC 

13 

6.85 

8% 

15% 

23% 

15% 

38% 

Deployed  SO  Unit 

27 

7.19 

11% 

7% 

15% 

26%  41%  I 

AFSOC 

2 

2.50 

50% 

50% 

0% 

0% 

0% 

MARSOC 

3 

3.67 

0% 

67%  [ 

33% 

0% 

0%I 

USASOC 

264 

7.64 

6% 

10% 

10% 

25% 

50% 

CA 

30 

7.30 

7% 

7% 

10% 

43% 

33% 

MISG 

55 

7.31 

2% 

15% 

15% 

29% 

40% 

SF 

152 

8.14 

5% 

7% 

8% 

20% 

61%  I 

Note.  1  =  Less  than  10%,  2  =  10-20%,  3  =  21-30%,  4  =  31-40%,  5  =  41-50%,  6  =  51-60%,  7  =  61-70%,  8  =  71-80%,  9  =  81-90%,  10  =  91-100%.  For  the  responses  frequencies,  responses  to  1  and  2 
were  grouped  together,  3  and  4  together,  5  and  6  together,  7  and  8  together,  and  9  and  10  together.  However,  means  are  based  on  the  1-10  point  scale.  Army  SOF  types  did  not  significantly  differ. 
Differing  sample  sizes  prevented  SOF  component  comparisons.  No  WARCOM  leaders  responded  to  this  item. 


Appendix  C,  Table  4.  CLPMs:  Estimated  percentage  of  SOF  operators  up-to-date  on  language  testing  requirement 


Group 

n 

M 

0-20% 

21-40% 

41-60% 

61-80% 

81-100% 

CLPMs 

10 

6.30 

20%  [ 

0% 

20% 

30% 

30% 

Note.  1  =  Less  than  10%o,  2  =  10-20%,  3  =  21-30%,  4  =  31-40%,  5  =  41-50%,  6  =  51-60%,  7  =  61-70%,  8  =  71-80%,  9  =  81-90%,  10  =  91-100%.  For  the  responses  frequencies,  responses  to  1  and  2 
were  grouped  together,  3  and  4  together,  5  and  6  together,  7  and  8  together,  and  9  and  10  together.  However,  means  are  based  on  the  1-10  point  scale. 
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APPENDIX  D:  DLPT  PERCEPTIONS  -  SOF  OPERATORS 

Appendix  D,  Table  1.  DLPT’s  relatedness  to  what  SOF  operators  do  on  the  job 


Group 

n 

M 

Not  Related  Slightly  Related 

Moderately  Related 

Re  late  d  Ve  ry  Re  late  d 

Overall 

1,058 

2.27 

31% 

30% 

23% 

n% 

4% 

USSOCOM  HQ 

113 

2.28 

28% 

33%  1 

24% 

12% 

3% 

JSOC 

1 

1.00 

100% 

0%  1 

0% 

0% 

0% 

TSOC 

16 

2.13 

31% 

25% 

44% 

0% 

0% 

Deployed  SO  Unit 

47 

2.15 

32% 

34% 

23%  9% 

2% 

AFSOC 

13 

2.92 

0% 

38% 

31% 

31% 

0% 

WARCOM 

6 

2.67 

0% 

50% 

33% 

17% 

0% 

MARSOC 

9 

2.56 

33% 

11% 

22% 

33% 

0% 

USASOC 

782 

2.25 

33% 

29% 

23% 

11% 

5% 

CA 

152 

2.74x 

22% 

20% 

30% 

18% 

10% 

MISG 

157 

2.43y 

26% 

29% 

27% 

10% 

7% 

SF 

466 

2.03z 

38% 

32% 

19% 

9% 

2% 

Note.  1  =  Not  Related,  2  =  Slightly  Related,  3  =  Moderately  Related,  4  =  Related,  5  =  Very  Related.  Army  SOF  types  sharing  the  same  letter  (e.g.,  x  or  y)  did  not  report  significantly  different  responses. 
Differing  sample  sizes  prevented  SOF  component  comparisons. 
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Appendix  D,  Table  2.  The  content  of  the  DLPT  is  clearly  related  to  what  SOF  operators  do  during  deployment 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree  Disagree 

Disagree 

Agree  Strongly  Agree 

Overall 

1,036 

2.40 

24% 

28% 

35% 

12% 

2% 

USSOCOM  HQ 

109 

2.57 

20% 

26% 

33% 

19% 

2% 

JSOC 

1 

3.00 

0% 

0% 

100% 

0% 

0% 

TSOC 

15 

2.40 

27% 

13%  1 

53% 

7% 

0% 

Deployed  SO  Unit 

47 

2.19 

30% 

30% 

32% 

9% 

0% 

AFSOC 

13 

3.15 

0% 

23% 

46% 

23% 

8% 

WARCOM 

6 

2.67 

0% 

50% 

33% 

17% 

0% 

MARSOC 

8 

2.25 

25% 

25% 

50% 

0% 

0% 

USASOC 

768 

2.37 

25% 

28% 

34% 

10% 

2% 

CA 

149 

2.77x 

13% 

26% 

38% 

19% 

5% 

MISG 

152 

2.49x 

19% 

27% 

41% 

12% 

1% 

SF 

460 

2.20y 

31% 

29% 

32% 

7% 

2% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Army  SOF  types  sharing  the  same  letter  (e.g.,  x  or  y)  did  not  report  significantly  different 
responses.  Differing  sample  sizes  prevented  SOF  component  comparisons. 
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Appendix  D,  Table  3.  SOF  operators’  DLPT  ratings  accurately  reflect  their  ability  to  use  the  language  on  the  job 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree  Disagree 

Disagree 

Agree  Strongly  Agree 

Overall 

1,037 

2.60 

20% 

25% 

33% 

19% 

4%| 

USSOCOM  HQ 

109 

2.68 

21% 

18% 

38% 

17% 

6% 

JSOC 

1 

3.00 

0% 

0% 

100% 

0% 

0% 

TSOC 

15 

2.67 

7% 

40% 

33% 

20% 

0% 

Deployed  SO  Unit 

47 

2.34 

26% 

30% 

30% 

15% 

0% 

AFSOC 

13 

3.46 

0% 

15% 

38% 

31% 

15% 

WARCOM 

6 

3.00 

0% 

33% 

33% 

33% 

0% 

MARSOC 

8 

2.25 

25% 

38% 

25% 

13% 

0% 

USASOC 

769 

2.58 

21% 

25% 

33% 

18% 

4%| 

CA 

149 

2.93x 

13% 

20% 

32% 

29% 

5% 

MISG 

153 

2.65xy 

18% 

24% 

35% 

20% 

3% 

SF 

460 

2.45y 

24% 

27% 

32% 

14% 

3% 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Army  SOF  types  sharing  the  same  letter  (e.g.,  x  or  y)  did  not  report  significantly  different 
responses.  Differing  sample  sizes  prevented  SOF  component  comparisons. 
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Appendix  D,  Table  4.  SOF  operators  who  score  well  on  the  DLPT  are  more  likely  to  successfully  use  language  in  the  field  than  those  who  score 
poorly 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree  Disagree 

Disagree 

Agree  Strongly  Agree 

Overall 

1,034 

2.99 

12% 

19% 

35% 

26% 

7% 

USSOCOM  HQ 

109 

3.01 

10% 

18% 

39% 

27% 

6%l 

JSOC 

1 

3.00 

0% 

0% 

100% 

O 

Vp 

O 

TSOC 

15 

2.67 

20% 

33% 

13% 

27% 

1% 

Deployed  SO  Unit 

47 

2.85 

13% 

26% 

30% 

28% 

4% 

AFSOC 

13 

3.39 

0% 

15% 

46%  23% 

15% 

WARCOM 

6 

3.00 

0% 

17%  1 

67% 

17% 

0% 

MARSOC 

7 

3.29 

14% 

14% 

29% 

14% 

29% 

USASOC 

767 

2.97 

12% 

19% 

35% 

26% 

7% 

CA 

148 

3.12 

9% 

17% 

37% 

26% 

11% 

MISG 

153 

3.12 

10% 

14% 

40% 

28% 

8% 

SF 

459 

2.88 

13% 

22% 

33% 

25% 

6% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Army  SOF  types  did  not  significantly  differ.  Differing  sample  sizes  prevented  SOF 
component  comparisons. 
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Appendix  D,  Table  5.  SOF  operators’  DLPT  ratings  accurately  reflect  their  ability  to  do  job/mission-related  tasks  in  the  target  language 


Neither  Agree  nor 

Group_ n  M_ Strongly  Disagree_ Disagree_ Disagree_ Agree_ Strongly  Agree 


Overall 

1,036 

2.61 

18% 

27% 

33% 

18% 

4%| 

USSOCOM  HQ 

109 

2.75 

14% 

29% 

30% 

21% 

6% 

JSOC 

1 

2.00 

0% 

100% 

0% 

0% 

0% 

TSOC 

15 

2.60 

7% 

47% 

27% 

20% 

0% 

Deployed  SO  Unit 

47 

2.40 

26% 

28% 

32% 

11%  I  4%i 

AFSOC 

13 

3.46 

0% 

8% 

54% 

23% 

15% 

WARCOM 

6 

3.00 

0% 

33% 

33% 

33% 

0% 

MARSOC 

8 

2.38 

25% 

25% 

38% 

13% 

0% 

USASOC 

768 

2.57 

20% 

26%  I 

33% 

17% 

3%| 

CA 

149 

2.85x 

15% 

19% 

37% 

23% 

5% 

MISG 

153 

2.75x 

13% 

27% 

36% 

21% 

3% 

SF 

459 

2.43y 

24% 

28% 

31% 

14% 

2% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Army  SOF  types  sharing  the  same  letter  (e.g.,  x  or  y)  did  not  report  significantly  different 
responses.  Differing  sample  sizes  prevented  SOF  component  comparisons. 
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Appendix  D,  Table  6.  There  are  language  tasks  SOF  operators  need  to  do  when  deployed  not  covered  by  the  DLPT 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree  Disagree 

Disagree 

Agree  Strongly  Agree 

Overall 

1,036 

3.67 

6% 

6%l 

29% 

35% 

25% 

USSOCOM  HQ 

108 

3.73 

4% 

6% 

30% 

36% 

25% 

JSOC 

1 

4.00 

0% 

0% 

0% 

100% 

0% 

TSOC 

15 

3.93 

0% 

7% 

20% 

47% 

27% 

Deployed  SO  Unit 

47 

3.85 

2% 

4% 

32% 

30% 

32% 

AFSOC 

13 

3.85 

0% 

0% 

46% 

23% 

31% 

WARCOM 

6 

4.17 

0% 

0% 

33% 

17% 

50% 

MARSOC 

8 

3.13 

25% 

0% 

38% 

13% 

25% 

USASOC 

770 

3.62 

6% 

6%| 

30% 

34% 

23% 

CA 

150 

3.66 

4% 

6% 

33% 

33% 

23% 

MISG 

153 

3.70 

4% 

5% 

32% 

35% 

24% 

SF 

460 

3.57 

8% 

7% 

28% 

34% 

23% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Army  SOF  types  did  not  significantly  differ.  Differing  sample  sizes  prevented  SOF 
component  comparisons. 
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APPENDIX  E:  DLPT  PERCEPTIONS  -  SOF  LEADERS 

Appendix  E,  Table  1.  DLPT’s  relatedness  to  what  SOF  operators  do  on  the  job 


Group 

n 

M 

Not  Related  Slightly  Related 

Moderately  Related 

Re  late  d  Ve  ry  Re  late  d 

Overall 

368 

2.57 

25% 

28% 

20% 

21% 

7% 

USSOCOM  HQ 

43 

2.35 

35% 

21% 

21% 

21% 

2% 

JSOC 

1 

4.00 

0% 

0%| 

0% 

100% 

0% 

TSOC 

13 

2.62 

23% 

31%  1 

15% 

23% 

8% 

Deployed  SO  Unit 

27 

2.70 

22% 

30% 

19% 

15% 

15% 

AFSOC 

2 

3.00 

50% 

0% 

0% 

0% 

50% 

MARSOC 

3 

3.00 

0% 

0% 

100% 

0% 

0% 

USASOC 

272 

2.57 

25% 

29% 

19% 

21% 

7% 

CA 

31 

3.39x 

3% 

16% 

29% 

42% 

10% 

MISG 

57 

2.98x 

16% 

23% 

21% 

28% 

12% 

SF 

156 

2.25y 

32% 

33% 

16% 

17% 

3% 

Note.  1  =  Not  Related,  2  =  Slightly  Related,  3  =  Moderately  Related,  4  =  Related,  5  =  Very  Related.  Army  SOF  types  sharing  the  same  letter  (e.g.,  x  or  y)  did  not  report  significantly  different  responses. 
Differing  sample  sizes  prevented  SOF  component  comparisons.  No  WARCOM  leaders  responded  to  this  item. 
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Appendix  E,  Table  2.  The  content  of  the  DLPT  is  clearly  related  to  what  SOF  operators  do  during  deployment 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree  Disagree 

Disagree 

Agree  Strongly  Agree 

Overall 

363 

2.32 

22% 

40% 

23% 

13% 

1%I 

USSOCOM  HQ 

42 

2.31 

31% 

31% 

17% 

19% 

2% 

JSOC 

1 

4.00 

0% 

0% 

0% 

100% 

0% 

TSOC 

14 

2.71 

14% 

29% 

29% 

29% 

0% 

Deployed  SO  Unit 

26 

2.50 

12% 

38% 

38% 

12% 

0% 

AFSOC 

2 

2.00 

0% 

100% 

0% 

0% 

0%| 

MARSOC 

3 

3.33 

0% 

33% 

0% 

67% 

0% 

USASOC 

268 

2.25 

23% 

43% 

23%  10% 

1% 

CA 

31 

2.77x 

3% 

39% 

39% 

16% 

3% 

MISG 

55 

2.60x 

9% 

42% 

33% 

13% 

4% 

SF 

154 

1.99y 

32% 

44% 

18% 

6% 

0% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Army  SOF  types  sharing  the  same  letter  (e.g.,  x  or  y)  did  not  report  significantly  different 
responses.  Differing  sample  sizes  prevented  SOF  component  comparisons.  No  WARCOM  leaders  responded  to  this  item. 
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Appendix  E,  Table  3.  SOF  operators’  DLPT  ratings  accurately  reflect  their  ability  to  use  the  language  on  the  job 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree  Disagree 

Disagree 

Agree  Strongly  Agree 

Overall 

362 

2.49 

19% 

37% 

24% 

19% 

2% ! 

USSOCOM  HQ 

42 

2.36 

26% 

33% 

21% 

■ 

17% 

2% 

JSOC 

1 

4.00 

0% 

0% 

0% 

100% 

0% 

TSOC 

14 

2.79 

7% 

36%  1 

29% 

29% 

0% 

Deployed  SO  Unit 

26 

2.39 

15% 

46% 

23% 

1 

15% 

0% 

AFSOC 

2 

2.00 

0% 

100% 

0% 

0% 

0% 

MARSOC 

3 

2.67 

0% 

67%  [ 

0% 

33% 

0% 

USASOC 

267 

2.48 

19% 

36%  1 

25% 

18% 

2% 

CA 

30 

2.80 

10% 

37% 

20% 

30% 

3% 

MISG 

55 

2.71 

9% 

36% 

33% 

18% 

4% 

SF 

154 

2.33 

25% 

36% 

22% 

16% 

1% 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Army  SOF  types  did  not  significantly  differ.  Differing  sample  sizes  prevented  SOF 
component  comparisons.  No  WARCOM  leaders  responded  to  this  item. 
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Appendix  E,  Table  4.  SOF  operators  who  score  well  on  the  DLPT  are  more  likely  to  successfully  use  language  in  the  field  than  those  who  score 
poorly 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree  Disagree 

Disagree 

Agree 

Strongly  Agree 

Overall 

363 

3.36 

7% 

13%  1 

24% 

47% 

8%| 

USSOCOM  HQ 

42 

3.31 

10% 

12% 

26% 

43% 

10%  1 

JSOC 

1 

4.00 

0% 

0% 

0% 

100% 

0% 

TSOC 

14 

3.71 

7% 

7% 

7% 

64% 

14%  [ 

Deployed  SO  Unit 

26 

3.73 

0% 

12% 

8% 

77% 

4%| 

AFSOC 

2 

2.50 

0% 

50% 

50%  0% 

0%[ 

MARSOC 

3 

3.67 

0% 

0% 

33% 

67% 

0% 

USASOC 

268 

3.30 

7% 

14% 

27%  44% 

7%| 

CA 

31 

3.45 

6% 

16% 

13% 

55% 

10% 

MISG 

55 

3.44 

5% 

11% 

25% 

51% 

7% 

SF 

154 

3.20 

8% 

17% 

30% 

38% 

7% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Army  SOF  types  did  not  significantly  differ.  Differing  sample  sizes  prevented  SOF 
component  comparisons.  No  WARCOM  leaders  responded  to  this  item. 
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Defense  Language  Proficiency  Test  (DLPT) 


Appendix  E,  Table  5.  SOF  operators’  DLPT  ratings  accurately  reflect  their  ability  to  do  job/mission-related  tasks  in  the  target  language 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree  Disagree 

Disagree 

Agree  Strongly  Agree 

Overall 

361 

2.59 

14% 

38%  I 

26% 

20% 

2% 

USSOCOM  HQ 

42 

2.31 

26% 

36%  1 

19% 

19% 

0% 

JSOC 

1 

4.00 

0% 

0%l 

0% 

100% 

0% 

TSOC 

14 

2.93 

7% 

29%  I 

29% 

36% 

0% 

Deployed  SO  Unit 

26 

2.92 

0% 

38%  I 

38% 

15% 

8% 

AFSOC 

2 

1.50 

50% 

50%  1 

0% 

0% 

0% 

MARSOC 

3 

3.00 

0% 

33%  1 

33% 

u 

33% 

0% 

USASOC 

266 

2.57 

14% 

39%  1 

26% 

19% 

2% 

CA 

31 

3.00x 

10% 

19%  1 

39% 

26% 

6% 

MISG 

55 

2.73xy 

9% 

35% 

33% 

22% 

2% 

SF 

152 

2.42y 

16% 

46% 

20% 

17% 

1% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Army  SOF  types  sharing  the  same  letter  (e.g.,  x  or  y)  did  not  report  significantly  different 
responses.  Differing  sample  sizes  prevented  SOF  component  comparisons.  No  WARCOM  leaders  responded  to  this  item. 
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Defense  Language  Proficiency  Test  (DLPT) 


Appendix  E,  Table  6.  There  are  language  tasks  SOF  operators  need  to  do  when  deployed  not  covered  by  the  DLPT 


Neither  Agree  nor 


Group 

n 

M 

Strongly  Disagree  Disagree 

Disagree 

Agree  Strongly  Agree 

Overall 

363 

4.10 

2% 

4% 

14% 

43% 

37% 

USSOCOM  HQ 

42 

4.24 

0% 

5% 

17% 

29% 

50% 

JSOC 

1 

4.00 

0% 

0% 

0% 

100%  0%l 

TSOC 

14 

4.00 

7% 

0% 

7% 

57% 

29% 

Deployed  SO  Unit 

26 

4.23 

0% 

0% 

8% 

62% 

31%  1 

AFSOC 

2 

4.50 

0% 

0% 

0% 

50% 

50% 

MARSOC 

3 

3.67 

0% 

33% 

0% 

33% 

33% 

USASOC 

268 

4.07 

2% 

4% 

15% 

43% 

36%  [ 

CA 

31 

3.84 

0% 

10% 

16% 

55% 

19% 

MISG 

55 

4.02 

0% 

2% 

25% 

42% 

31% 

SF 

154 

4.13 

3% 

4% 

10% 

42% 

41% 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree.  Army  SOF  types  did  not  significantly  differ.  Differing  sample  sizes  prevented  SOF 
component  comparisons.  No  WARCOM  leaders  responded  to  this  item. 
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SOF  Language  and  Culture  Needs  Assessment  Project 


Defense  Language  Proficiency  Test  (DLPT) 


APPENDIX  F:  DLPT  PERCEPTIONS  -  STUDENTS  IN  THE  SOF  TRAINING  PIPELINE 


Appendix  F,  Table  1.  DLPT’s  relatedness  to  what  SOF  operators  do  on  the  job 


Group 

n 

M 

Not  Related 

Slightly  Related 

Mode  rate  ly  Re  late  d 

Related 

Very  Related 

Pipeline 

27 

2.37 

33%  1 

26%  1 

19% 

15%  l 

7%l 

Note.  1  =  Not  Related ,  2  =  Slightly  Related,  3  =  Moderately  Related,  4  =  Related,  5  =  Very  Related. 


Appendix  F,  Table  2.  The  content  of  the  DLPT  is  clearly  related  to  what  SOF  operators  do  during  deployment 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

Pipeline 

25 

2.80 

12%  1 

20% 

44% 

24%  I 

0% 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  F,  Table  3.  SOF  operators’  DLPT  ratings  accurately  reflect  their  ability  to  use  the  language  on  the  job 


Group 

n 

M 

Strongly  Disagree  Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

Pipeline 

25 

2.84 

8%1  32%  ■ 

28%  ■ 

32%  l 

0% 

Note.  1  =  Strongly  Disagree,  2  = 

Disagree,  3 

=  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 

Appendix  F,  Table  4.  SOF  operators  who  score  well  on  the  DLPT  are  more 

likely  to  successfully  use 

language  in  the  field  than  those  who  score 

poorly 

Neither  Agree  nor 

Group 

n 

M 

Strongly  Disagree  Disagree 

Disagree 

Agree 

Strongly  Agree 

Pipeline 

25 

3.48 

4%|  4%| 

36% 

52%  I 

4% 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 
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Appendix  F,  Table  5.  SOF  operators’  DLPT  ratings  accurately  reflect  their  ability  to  do  job/mission-related  tasks  in  the  target  language 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

Pipeline 

25 

2.88 

8%[ 

28%  l 

32% 

32%  I 

0% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  F,  Table  6.  There  are  language  tasks  SOF  operators  need  to  do  when  deployed  not  covered  by  the  DLPT 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

Pipeline 

25 

3.72 

4%  I 

0%l 

32% 

48% 

16%  1 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 
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SOF  Language  and  Culture  Needs  Assessment  Project  Defense  Language  Proficiency  Test  (DLPT) 

APPENDIX  G:  DLPT  PERCEPTIONS  -  MI  LINGUISTS  ASSIGNED  TO  A  SOF  UNIT 


Appendix  G,  Table  1.  DLPT’s  relatedness  to  what  MI  Linguists  do  on  the  job 


Group  n 

M 

Not  Related 

Slightly  Related 

Mode  rate  ly  Re  late  d 

Related 

Very  Related 

MI  Linguists  or  09L  62 

2.74 

27%  1 

21%  1 

15%  1 

24%  I 

13%  1 

Note.  1  =  Not  Related ,  2  =  Slightly  Related ,  3  =  Moderately  Related ,  4  =  Related ,  5  =  Very  Related. 


Appendix  G,  Table  2.  The  content  of  the  DLPT  is  clearly  related  to  what  MI  Linguists  do  during  deployment 


Group  n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

MI  Linguists  or  09L  60 

2.70 

23%  1 

23% 

18% 

30% 

5%[ 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  G,  Table  3.  MI  Linguists’  DLPT  ratings  accurately  reflect  their  ability  to  use  the  language  on  the  job 


Group  n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

MI  Linguists  or  09L  60 

2.88 

22%  1 

20% 

17%  i 

32%  | 

10%  1 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  G,  Table  4.  SOF  operators  who  score  well  on  the  DLPT  are  more  likely  to  successfully  use  language  in  the  field  than  those  who  score 
poorly 


Group  n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

MI  Linguists  or  09L  60 

3.10 

10%[ 

20% 

27% 

37% 

7% 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 
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Appendix  G,  Table  5.  MI  Linguists’  DLPT  ratings  accurately  reflect  their  ability  to  do  job/mission-related  tasks  in  the  target  language 


Group  n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

MI  Linguists  or  09L  60 

2.88 

20%  1 

22%  [ 

18%  I 

30% 

10% 

Note.  1  =  Strongly  Disagree ,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  G,  Table  6.  There  are  language  tasks  MI  Linguists  need  to  do  when  deployed  not  covered  by  the  DLPT 


Group  n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

MI  Linguists  or  09L  60 

4.00 

2%| 

7%l 

12% 

50% 

30%  L 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 
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SOF  Language  and  Culture  Needs  Assessment  Project 


Defense  Language  Proficiency  Test  (DLPT) 


APPENDIX  H:  DLPT  PERCEPTIONS  -  CLPMs 


Appendix  G,  Table  1.  DLPT’s  relatedness  to  what  SOF  operators  do  on  the  job 


Group 

n 

M 

Not  Related 

Slightly  Related 

Moderately  Related 

Related 

Very  Related 

CLPMs 

10 

2.60 

30%  1 

20% 

20% 

20% ! 

10% 

Note.  1  =  Not  Related ,  2  =  Slightly  Related,  3  =  Moderately  Related,  4  =  Related,  5  =  Very  Related. 


Appendix  G,  Table  2.  The  content  of  the  DLPT  is  clearly  related  to  what  SOF  operators  do  during  deployment 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

CLPMs 

10 

2.10 

30%  I 

30% 

40%  I 

0%| 

0%| 

Note.  1  =  Strongly  Disagree,  2  = 

Disagree,  3  =  Neither  Agree  nor  Disagree,  4 

=  Agree,  5  =  Strongly  Agree. 

Appendix  G,  Table  3.  SOF  operators’  DLPT  ratings  accurately  reflect  their  ability  to  use  the  language 

on  the  job 

Neither  Agree  nor 

Group 

n 

M 

Strongly  Disagree 

Disagree 

Disagree 

Agree 

Strongly  Agree 

CLPMs 

10 

2.60 

40%  [ 

10% 

0%l 

50% 

0%l 

Note.  1  =  Strongly  Disagree,  2  = 

Disagree,  3  =  Neither  Agree  nor  Disagree,  4 

=  Agree,  5  =  Strongly  Agree. 

Appendix  G,  Table  4.  SOF  operators  who  score  well  on  the  DLPT  are  more  likely  to  successfully  use 

language  in  the  field  than  those  who  score 

poorly 

Neither  Agree  nor 

Group 

n 

M 

Strongly  Disagree 

Disagree 

Disagree 

Agree 

Strongly  Agree 

CLPMs 

10 

3.00 

20%  [ 

20% 

10%| 

40% 

10%  1 

Note.  1  =  Strongly  Disagree,  2  = 

Disagree,  3  =  Neither  Agree  nor  Disagree,  4 

=  Agree,  5  =  Strongly  Agree. 
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Defense  Language  Proficiency  Test  (DLPT) 


Appendix  G,  Table  5.  SOF  operators’  DLPT  ratings  accurately  reflect  their  ability  to  do  job/mission-related  tasks  in  the  target  language 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

CLPMs 

9 

2.56 

33%  1 

11% 

33% 

11%I 

11% 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 


Appendix  G,  Table  6.  There  are  language  tasks  SOF  operators  need  to  do  when  deployed  not  covered  by  the  DLPT 


Group 

n 

M 

Strongly  Disagree 

Disagree 

Neither  Agree  nor 
Disagree 

Agree 

Strongly  Agree 

CLPMs 

9 

4.33 

0%l 

11% 

11% 

11%I 

67%  ■ 

Note.  1  =  Strongly  Disagree,  2  =  Disagree,  3  =  Neither  Agree  nor  Disagree,  4  =  Agree,  5  =  Strongly  Agree. 
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SOF  Language  and  Culture  Needs  Assessment  Project 


Defense  Language  Proficiency  Test  (DLPT) 


APPENDIX  Is  SURVEY  COMMENT  TOTALS  BY  DLPT  VERSION 

Appendix  I,  Table  1.  Survey  Comment  totals  -  DLPT5  only 


Theme 

Percentage  of  Total  Comments 

SOF  Operators 

SOF  Leaders 

Combined 

DLPT  is  an  accurate/valid  assessment 

1.1% 

1.1% 

1.1% 

General  positive 

- 

0.5% 

0.2% 

Test  content/format/protocol  is  good/appropriate 

- 

0.5% 

0.2% 

Training  matches  what  is  tested  on  the  DLPT 

- 

- 

- 

DLPT  content  is  related  to  mission/job/military 

- 

0.5% 

0.2% 

DLPT  content  is  unrelated  to  mission/job/military 

2.5% 

2.6% 

2.5% 

DLPT  is  not  an  accurate/valid  assessment 

5.7% 

6.3% 

5.9% 

Needs  to  include  a  speaking  component 

1.4% 

- 

0.8% 

Test  content/format/protocol  is  bad/not  appropriate 

1.1% 

3.2% 

1.9% 

DLPT  is  too  difficult 

3.5% 

5.3% 

4.2% 

Leadership/policy  problems  related  to  DLPT  testing 

1.8% 

1.6% 

1.7% 

Needs  to  cover  more  military-specific  topics 

0.7% 

1.1% 

0.8% 

General  negative 

- 

1.6% 

0.6% 

Training  does  not  match  what  is  tested  on  the  DLPT 

0.4% 

1.1% 

0.6% 

Not  able  to  prepare  for  the  test 

- 

- 

- 

Total  Comments 

282 

189 

471 
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Appendix  I,  Table  2.  Survey  Comment  totals  -  General  Responses 


Theme 

Percentage  of  Total  Comments 

SOF  Operators 

SOF  Leaders 

Combined 

DLPT  content  is  unrelated  to  mission/job/military 

20.9% 

22.8% 

21.7% 

DLPT  is  not  an  accurate/valid  assessment 

9.9% 

12.7% 

11.0% 

Needs  to  include  a  speaking  component 

10.6% 

15.3% 

12.5% 

Test  content/format/protocol  is  bad/not  appropriate 

8.9% 

12.2% 

10.2% 

DLPT  is  too  difficult 

4.3% 

5.3% 

4.7% 

Leadership/policy  problems  related  to  DLPT  testing 

4.3% 

10.6% 

6.8% 

Needs  to  cover  more  military-specific  topics 

7.1% 

7.4% 

7.2% 

General  negative 

3.5% 

9.0% 

5.7% 

Training  does  not  match  what  is  tested  on  the  DLPT 

3.9% 

2.6% 

3.4% 

Not  able  to  prepare  for  the  test 

1.1% 

2.1% 

1.5% 

DLPT  is  an  accurate/valid  assessment 

2.1% 

5.3% 

3.4% 

General  positive 

0.7% 

1.6% 

1.1% 

Test  content/format/protocol  is  good/appropriate 

0.7% 

1.1% 

0.8% 

Training  matches  what  is  tested  on  the  DLPT 

0.7% 

0.5% 

0.6% 

DLPT  content  is  related  to  mission/job/military 

- 

- 

- 

Total  Comments 

282 

189 

471 
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APPENDIX  J:  COMMENT  CODE  DEFINITIONS 

SOF  operators  and  leaders  provided  comments  to  the  following  prompts: 

•  Responses  for  “Other”  problems  experienced  during  DLPT  testing. 

•  Use  the  space  below  to  provide  any  specific  feedback  that  you  have  related  to  issues  you 
experienced  while  taking  the  DLPT. 

•  Please  provide  any  specific  feedback  you  have  related  to  the  DLPT. 

All  comments  were  content  analyzed  to  extract  common  themes.  The  resulting  themes  are  provided 
below  by  item  prompt  with  a  definition  and  verbatim  exemplar  comments  that  illustrate  the  theme.  For 
more  information  about  this  study’s  content  analysis  process,  please  refer  to  the  Methodology  Report 
(Technical  Report  #201001 1002). 

Note:  Exemplar  comments  are  presented  verbatim  and  are  uncorrected  for  spelling  and  other  mistakes. 

Responses  for  “Other”  problems  experienced  during  DLPT  testing.27 

•  Test  content/format/protocol  problems 

o  Definition:  Respondents  commented  on  the  test  structure,  content  (e.g.,  questions, 
answers),  or  process. 

■  “English  answers  doen't  make  sense  some  time” 

•  Poor  audio  quality 

o  Definition:  Respondents  commented  on  audio  problems  during  testing. 

■  “hearing  the  muffled  cassette  recording” 

•  Testing  environment  problems 

o  Definition:  Respondents  commented  on  testing  environment  conditions  (e.g., 
temperature,  noise)  or  disruptions. 

■  “Poor  testing  environment  with  no  AC  in  the  spring  and  summer” 

•  DLPT  is  not  job  relevant 

o  Definition:  Respondents  commented  that  the  DLPT  does  not  relate  to  what  they  do 
on  the  job  or  while  deployed. 

■  “test  having  no  relivancy  to  what  is  needed  while  deployed” 

•  Not  able  to  prepare  for  the  test 

o  Definition:  Respondents  commented  that  they  were  not  able  to  prepare  for  the  DLPT 
due  to  other  training  and/or  job  requirements. 

■  “no  prep  time” 

•  General  negative 

o  Definition:  Respondents  provided  non-favorable  and  non-specific  comments  on  the 
DLPT. 

■  “DLPT  V  Sucks.” 


27  Only  SOF  operators  received  this  item. 
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•  Scheduling  conflicts 

o  Definition:  Respondents  commented  that  they  had  difficulty  scheduling  the  DLPT 
due  to  other  training,  other  job  requirements,  and/or  limited  availability. 

■  “OPTEMPO  issues” 

•  Scoring  system  is  not  fair/accurate 

o  Definition:  Respondents  commented  that  they  do  not  believe  the  DLPT  is  scored  in  a 
fair  or  accurate  way.  Also  included  comments  that  the  score  did  not  accurately 
reflect  their  language  ability. 

■  “DLPT  V  grading  system  is  unrealistic.” 

Use  the  space  below  to  provide  any  specific  feedback  that  you  have  related  to  issues  you 
experienced  while  taking  the  DLPT.28 

•  None/NA/not  relevant  response 

o  Definition:  Respondents  had  no  comment  or  made  an  unrelated  comment. 

■  “NONE  AT  THIS  TIME” 

•  Test  content/format/protocol  is  bad/not  appropriate 

o  Definition:  Respondents  provided  non- favorable  comments  on  the  test  structure, 
content  (e.g.,  questions,  answers),  or  process. 

■  “Besides  the  fact  that  the  questions  and  answers  are  left  open  to  too  much 
interpretation  and  you  are  left  guessing  what  the  test  writters  are  trying  to 
say,  the  tests  need  to  be  reviewed  by  someone  not  interested  in  a  contract  and 
making  money  off  of  the  Government.  The  test  is  unnecessarily  and 
extremely  too  long.  The  questions  are  not  clear  and  the  answers  are  not 
obvious,  leaving  one  to  have  to  guess  the  answer  the  test  is  asking  for... the 
questions  are  often  too  vague!  The  use  of  slang  in  the  target  languages  and 
the  speakers  being  of  numerous  national  origens.  Spanish  is  not  Spanish  is 
not  Spanish  -  each  country  has  a  unique  way  of  speaking  and  often  times  the 
coloquilisms  are  very  different.  Portuguese  is  NOT  Protuguese  throughout 
the  world  either  -  they  are  given  as  different  tests  and  we  should  be  awarded 
FLPP  for  each.” 

•  Test  content/format/protocol  is  good/appropriate 

o  Definition:  Respondents  provided  favorable  comments  on  the  test  structure,  content 
(e.g.,  questions,  answers),  or  process. 

■  “The  administrators  are  professional  and  always  on  time  to  administer  the 
test.” 

•  T  echnical  problems 

o  Definition:  Respondents  commented  on  equipment  or  computer  problems  they 
encountered  during  testing. 

■  “The  computers  would  freeze  up  during  the  test  and  you  would  either  have  to 
change  computers  or  reboot  the  system.” 


28  Only  SOF  operators  received  this  item. 


11/12/10 


©  SWA  Consulting  Inc.,  2010 
Technical  Report  [2010011015] 


Page  59 


SOF  Language  and  Culture  Needs  Assessment  Project 


Defense  Language  Proficiency  Test  (DLPT) 


•  Scheduling  conflicts 

o  Definition:  Respondents  commented  that  they  had  difficulty  scheduling  the  DLPT 
due  to  other  training,  other  job  requirements,  and/or  limited  availability. 

■  “need  more  test  days  or  slots  on  a  day  to  test” 

•  DLPT  is  not  an  accurate  assessment 

o  Definition:  Respondents  commented  that  they  do  not  believe  the  DLPT  is  scored  in  a 
fair  or  accurate  way.  Also  included  comments  that  the  score  did  not  accurately 
reflect  their  language  ability. 

■  “The  MSA  DLPT  V  is  not  a  true  measure  of  a  students  language  proficiency 
in  MSA.  I  scored  a  2+/2+  on  the  DLPT  IV,  and  retested  on  the  DLPT  V  two 
days  later,  scoring  a  0/0.” 

•  Testing  environment  problems 

o  Definition:  Respondents  commented  on  testing  environment  conditions  (e.g., 
temperature,  noise)  or  disruptions. 

■  “too  many  people  in  test  area.... most  of  them  are  the  test  taker...  should  limit 
the  amount  of  people.  The  area  was  uncomfortable  due  to  the  location  and 
temperature” 

•  DLPT  is  not  job  relevant 

o  Definition:  Respondents  commented  that  the  DLPT  does  not  relate  to  what  they  do 
on  the  job  or  while  deployed. 

■  “DLPT  does  not  cover  any  material  which  is  relavent  to  my  job.  The  DLPT 
deals  mainly  with  art  and  culture,  not  with  military  or  government 
situations.” 

•  Leadership/policy  problems  related  to  DLPT  testing 

o  Definition:  Respondents  commented  on  leadership  and/or  policy  issues  related  to 
DLPT  testing,  such  as  requirements  and  protecting  time  to  study  for  and  take  the  test. 

■  “There  was  an  issue  with  scheduling  dates  to  take  the  DLPT  5  after  language 
immersion  instead  of  before  but  it  was  more  a  command  related  issue. 
Additionally  there  was  an  issue  with  getting  an  un-official  score  immediately 
after  due  to  an  issue  with  the  contract  for  the  testing  administrators.” 

•  DLPT  is  too  difficult 

o  Definition:  Respondents  commented  that  the  DLPT  was  too  difficult  or  that  it  was 
written  for  too  high  of  an  educational  level  (e.g.,  written  for  college  students  rather 
than  military  personnel). 

■  “There  is  a  dramatic  difference  between  the  DLPT  IV  and  DLPT  V.  The 
DLPT  IV  is  a  reasonable  standard  that  the  average  non-native  speaker  can 
master.  The  DLPT  V  is  a  professional  level  test  that  requires  college  level 
fluency  in  both  the  target  language  and  English.  There  were  native  speaking 
language  instructors  in  my  school  that  were  unable  to  correctly  answer 
questions  from  the  DLPT  V  sample  exam.  It  is  that  difficult.  I  don't  know 
why  the  level  of  difficulty  changed  so  dramatically,  but  I  believe  there  must 
be  some  motivation  for  it  because  a  change  that  drastic  is  not  accidental.” 
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•  General  negative 

o  Definition:  Respondents  provided  non- favorable  and  non-specific  comments  on  the 
DLPT. 

■  “the  dipt  5  is  crap” 

•  Training  does  not  match  what  is  tested  on  the  DLPT 

o  Definition:  Respondents  commented  that  the  training  they  received  in  either  initial 
acquisition  or  sustainment/enhancement  does  not  match  what  is  tested  on  the  DLPT. 

■  “The  DLPT  is  not  relevent  to  the  type  of  training  we  do.” 

•  Delays  getting  scores 

o  Definition:  Respondents  commented  that  they  experienced  delays  receiving  their 
DLPT  score  or  that  it  took  too  long  to  receive. 

■  “it  took  2  1/2  weeks  to  get  my  scores.” 

•  Not  receiving  test  feedback  for  improvement 

o  Definition:  Respondents  commented  that  they  do  not  receive  feedback  on  their  DLPT 
performance  to  use  for  improvement  on  their  next  test. 

■  “And  the  test  really  doesn't  give  you  any  feed  back  on  your  answers.” 

Please  provide  any  specific  feedback  you  have  related  to  the  DLPT. 

•  None/NA/not  relevant  response 

o  Definition:  Respondents  indicated  they  had  no  comment  or  made  an  unrelated 
comment. 

■  “nothing  to  say” 

•  DLPT  content  is  unrelated  to  mission/job/military 

o  Definition:  Respondents  commented  that  the  DLPT  does  not  relate  to  what  they  do 
on  the  job  or  while  deployed. 

■  “The  DLPT  should  put  more  emphasis  on  military  operations  and  training, 
politics,  and  mastery  of  military  terminology.  There  is  not  enough  "D"  in  the 
DLPT.  /  While  I  may  have  had  opportunities  to  talk  about  a  speech  given  at  a 
university  by  a  poet  or  a  soccer  game  in  the  capital  city,  it  has  never  really 
been  important  to  mission  accomplishment.” 

•  DLPT  content  is  related  to  mission/job/military 

o  Definition:  Respondents  commented  that  the  DLPT  relates  to  what  they  do  on  the  job 
or  while  deployed. 

■  “There  are  a  lot  of  proponents  for  taking  the  OPI  vs  DLPT  5  because  they 
state  an  operator  only  needs  to  be  able  to  communicate  in  the  target  language 
to  accomplish  the  mission.  I  disagree  with  this  way  of  thinking,  and  I  also 
believe  it  can  be  harmful  to  the  force  in  the  future.  [. . .]  the  DLPT  5  is  a  good 
measure  on  whether  an  operator  is  proficient  enough  to  get  along  in  the  target 
language  while  on  a  mission.” 

•  Needs  to  include  a  speaking  component 

o  Definition:  Respondents  commented  that  the  DLPT  lacks  a  speaking  section  or  that 
speaking  proficiency  needs  to  be  tested. 
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■  “The  problem  with  the  testing  is  that  it  does  not  include  a  speaking  portion. 
To  get  a  better  idea  of  the  Soldiers  language  capability,  there  should  be  an 
interview  portion  using  a  native  speaker  of  the  tested  language.” 

Test  content/format/protocol  is  bad/not  appropriate 

o  Definition:  Respondents  provided  non- favorable  comments  on  the  test  structure, 
content  (e.g.,  questions,  answers),  or  process. 

■  “The  DLPT  for  Spanish  is  a  grouping  of  many  dialects  as  well  as  a  number 
of  radio  broadcasts  that  are  difficult  for  the  native  and  non-native  Spanish 
speaker  to  grasp  in  the  testing  format.  E.g.  the  tester  will  hear  a  Uraguan 
native  speaking  followed  by  a  Puerto  Rican  -  very  different  dialects  of 
Spanish  with  significant  differences  in  idioms.  Jumping  from  one  dialect  to 
the  next  in  sequence  is  very  difficult  to  master  -  again,  both  for  the  native  and 
non-native  speaker.” 

Test  content/format/protocol  is  good/appropriate 

o  Definition:  Respondents  provided  favorable  comments  on  the  test  structure,  content 
(e.g.,  questions,  answers),  or  process. 

■  “Informal  language  is  not  covered  in  the  Indonesian  DLPT,  but  it  changes  so 
frequently  that  it  better  to  test  operators  on  the  ability  to  use  the  formal 
language.  The  lowest  common  denominator  will  always  be  the  formal 
version  of  Indonesian.” 

DLPT  is  not  an  accurate/valid  assessment 

o  Definition:  Respondents  commented  that  the  DLPT  does  not  accurately  assess 
language  proficiency  or  ability  to  use  the  language. 

■  “Compaired  with  previous  versions  of  the  DLPT  I  have  dramatically  lower 
confidence  in  the  validity  of  test  results  in  both  directions,  low  scores  no 
longer  mean  low  ability  and  high  scores  no  longer  mean  high  ability.  With 
out  having  investigated  or  tested  it  my  hypothesis  is  that  the  DLPT  5 
provides  more  random  results” 

DLPT  is  an  accurate/valid  assessment 

o  Definition:  Respondents  commented  that  the  DLPT  does  accurately  assess  language 
proficiency  or  ability  to  use  the  language. 

■  “DLPT  is  a  good  measure  of  skill” 

DLPT  is  too  difficult 

o  Definition:  Respondents  commented  that  the  DLPT  was  too  difficult  or  that  it  was 
written  for  too  high  of  an  educational  level  (e.g.,  written  for  college  students  rather 
than  military  personnel). 

■  “When  most  instructors  have  trouble  passing  a  test  that  most  students  must 
prepare  for,  there's  a  serious  problem.  This  Level  5  testing  has  made  an  SAT 
type  test  out  of  a  foreign  language  test  and  the  NS  A  calls  it  fair.  It's  very 
unfair  to  any  operator.  The  older  tests  weren't  ideal  either  but  at  least  there 
were  those  who  knew  the  language  well  enough  who  could  pass  it  and  get 
paid.  Now,  I  personally  know  several  former  3/3 's  who  are  getting  0+/1  or 
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1/1  on  the  level  5  Arabic  test.  They  can  speak  well  and  perform  their  jobs  as 
well  as  they  ever  could,  but  now  are  not  receiving  any  language  pay.” 

Leadership/policy  problems  related  to  DLPT  testing 

o  Definition:  Respondents  commented  on  leadership  and/or  policy  issues  related  to 
DLPT  testing,  such  as  requirements  and  protecting  time  to  study  for  and  take  the  test. 

■  “nobody  cares  about  it... we  test  in  a  language  that  we  do  not  use.... it  is  like 
me  asking  how  important  speaking  norwegian  is  to  you  and  how  much  time 
you  have  to  focus  on  that  language  knowing  you  will  never  go  to  norway” 

Needs  to  cover  more  military-specific  topics 

o  Definition:  Respondents  commented  that  the  DLPT  needs  to  include  more  military- 
specific  topics  and/or  scenarios. 

■  “I  dont  think  there  is  one  mentioin  of  any  type  of  military  training  on  the  test. 
[. . .]  We  are  afterall  a  military  training  foreign  military  units.  This  issue 
needs  to  be  fixed  to  become  relevant.” 

General  negative 

o  Definition:  Respondents  provided  non-favorable  and  non-specific  comments. 

■  “The  DLPT  (V)  is  the  worst  test  I  have  ever  taken.  [. . .]  DLPT  IV  was  a  big 
step  in  the  wrong  direction  that  has  resulted  in  DLPT  V.  I  will  not  take  an 
annual  DLPT  again  until  DLPT  V  is  gone.  It  is  a  waste  of  5  hours  of  my  life, 
and  it  is  a  waste  of  the  taxpayers  money  for  me  to  continue  to  valisate  that 
program  indirectly  by  participating  in  the  test.” 

General  positive 

o  Definition:  Respondents  provided  favorable  and  non-specific  comments. 

■  “The  DLPT  5  is  probably  a  better  test” 

Training  does  not  match  what  is  tested  on  the  DLPT 

o  Definition:  Respondents  commented  that  the  training  they  received  (e.g.,  initial 

acquisition  or  sustainment/enhancement)  does  not  match  what  is  tested  on  the  DLPT. 

■  “I  learned  language  through  SOLT.  The  vast  majority  of  the  language  from 
SOLT  does  not  cross  over  to  the  DLPT” 

Training  matches  what  is  tested  on  the  DLPT 

o  Definition:  Respondents  commented  that  the  training  they  received  (e.g.,  initial 
acquisition  or  sustainment/enhancement)  matches  what  is  tested  on  the  DLPT. 

■  “The  language  training  matches  the  DLPT.” 

Not  able  to  prepare  for  the  test 

o  Definition:  Respondents  commented  that  they  were  unable  to  prepare  for  the  DLPT 
due  to  other  training  and/or  job  requirements. 

■  “There  is  just  no  time  with  the  optemp  to  properly  study  for  the  DLPT” 
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